-- Logs begin at Wed 2022-08-17 18:52:16 UTC, end at Wed 2022-08-17 20:36:53 UTC. -- Aug 17 19:48:45.334537 dv-cinder-820027-19-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 17 19:48:45.342132 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 17 19:48:45 2022] *** Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: nodename: dv-cinder-820027-19-iscsi Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: machine: x86_64 Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: clock source: unix Aug 17 19:48:45.344149 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: pcre jit disabled Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: detected number of CPU cores: 4 Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: current working directory: / Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: detected binary path: /usr/bin/uwsgi-core Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: your processes number limit is 47680 Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: your memory page size is 4096 bytes Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: detected max file descriptor number: 2048 Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: lock engine: pthread robust mutexes Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: thunder lock: enabled Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 17 19:48:45.344571 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 17 19:48:45.386355 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: Python main interpreter initialized at 0x55d341c889c0 Aug 17 19:48:45.386355 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: python threads support enabled Aug 17 19:48:45.386355 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: your server socket listen backlog is limited to 100 connections Aug 17 19:48:45.386355 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: your mercy for graceful operations on workers is 90 seconds Aug 17 19:48:45.386768 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: mapped 403077 bytes (393 KB) for 2 cores Aug 17 19:48:45.386848 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: *** Operational MODE: preforking *** Aug 17 19:48:45.387104 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: *** uWSGI is running in multiple interpreter mode *** Aug 17 19:48:45.387104 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: spawned uWSGI master process (pid: 81516) Aug 17 19:48:45.387331 dv-cinder-820027-19-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 17 19:48:45.389179 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: spawned uWSGI worker 1 (pid: 81532, cores: 1) Aug 17 19:48:45.389179 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: spawned uWSGI worker 2 (pid: 81533, cores: 1) Aug 17 19:48:45.389179 dv-cinder-820027-19-iscsi devstack@keystone.service[81516]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 17 19:48:49.621833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:48:49.621833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Aug 17 19:48:49.622648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:48:49.622648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: from cryptography.utils import int_from_bytes Aug 17 19:48:49.627079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:48:49.627079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Aug 17 19:48:49.627542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 17 19:48:49.627542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: from cryptography.utils import int_from_bytes Aug 17 19:48:49.675881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 19:48:49.676111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 19:48:49.676197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] command line args: [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 19:48:49.676497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 19:48:49.676635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ================================================================================ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 19:48:49.676908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.677160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 17 19:48:49.677303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_dir = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.677303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.677434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 17 19:48:49.677503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] config_source = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.677595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] command line args: [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 17 19:48:49.677668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] debug = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.677823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 17 19:48:49.677922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ================================================================================ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 17 19:48:49.678204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_dir = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] config_source = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.678958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] debug = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_config_append = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_dir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.679963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_config_append = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_dir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.680945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.681838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.682959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.683946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] publish_errors = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.684935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] publish_errors = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] standard_threads = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.685908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_journal = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_json = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] standard_threads = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_stderr = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] use_syslog = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.686936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.687078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_journal = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.687316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_json = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_stderr = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.687594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] use_syslog = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.687844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 17 19:48:49.687985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.688959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.689947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.690977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.691943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.692942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.693883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.694836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.695944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.696952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.697884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.698884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.699918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.700982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.701838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.702922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.703966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.704901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.705867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.706923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.707956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.708896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.709988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.710988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.external = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.711948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.password = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.external = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.712906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] auth.token = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.713901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.password = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] auth.token = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.714870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.715969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.716883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.717992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.718940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.719889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.720906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.721967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.722971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.723904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.724962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.725995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.726859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.727960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.728966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.729980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.730992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.731928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.732967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.733947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.734949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.735989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.736985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.737908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.738882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.739921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.740903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.741919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.742941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.743931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.744954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.745904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.746918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.747917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748547 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.748915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.749908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.750892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.751929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.752995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.753952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.754940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.755956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.756900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.757900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.758972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.759942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.760948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.761681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81532) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 19:48:49.761828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.762060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.762303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.762618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.762864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.763108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.763355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.763591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.763833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.764097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.764302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.764536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.764770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.765014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.765250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.765498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.765760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.766006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.766241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.766484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.766715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.766958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.767193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 17 19:48:49.767355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 17 19:48:49.792085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.792497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.792587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.792721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.792852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.793057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.801438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.801780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.830197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.830423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.830766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.830899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.882518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.882978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.931703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.932009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.933101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.933346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.997573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.997802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.998036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.998254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.998473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:49.998689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.026379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.026676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.026908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.027131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.027344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.027554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.027770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.040402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.092361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.092565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.092667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.092854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.093095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.093293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.107659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.107879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.108053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.108296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.108485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.108731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.108894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.117404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.130647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.130889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.131075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.131265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.131510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.131722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.131899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81533) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.139434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.139664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.139887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.140108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.140326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.140539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.140751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81532) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 17 19:48:50.149080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.149283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:48:50.149659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.149857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:48:50.150128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.150383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.150719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.150896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.150985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:48:50.151158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.151248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.151443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 17 19:48:50.151782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.152048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.152308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.152568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.160609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.160886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.161156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.161441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.162031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.162312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.162583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.162874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.168429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.170241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.171667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.171949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.172231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.172510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.172793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.173319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.173932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.174997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.175270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.175538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.175789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.197136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.198367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.198671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.198953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.199936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.200225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.203650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.203934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.204215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.204752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.205037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.205326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.210787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.211076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.211818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.212108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.215237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.215527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.215810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.216366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.216928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.217014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.217172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.217299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.217577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.217869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.218127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.233672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.233956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.234462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.234737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.237813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.238724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.240882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.241178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.241919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.242210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.245507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.246388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.248646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.249497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.251729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.252033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.252322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.252722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.252850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.253850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.254134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.269035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.269335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.269648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.269920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.270362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.270655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.270920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.271186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.276587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.278185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.279480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.279712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.279845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.280059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.282415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.282602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.282784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.282878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.282954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.283173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.283315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.283533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.283663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.283805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.295211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.295413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.295509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.295697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.295790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.295974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.296942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.297071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.312142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.312433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.315070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.315570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.318094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.318455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.319528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.319854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.321016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.321307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.321433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.321604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.321756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.321885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.328105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.328331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.329576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.329801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.332441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.332593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.332784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.332917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.333055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.333239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.333333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.333455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.341236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.341525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.341739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.341836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.341965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.342194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.348547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81532) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.348949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.349092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81533) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 17 19:48:50.349285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.349417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.349580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.349727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.349993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.350985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.351257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.351454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.351595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.351742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.351875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.352015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.352156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.352340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.352504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81533) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.352640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.353098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.353594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.354074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81532) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 17 19:48:50.386868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d341c889c0 pid: 81533 (default app) Aug 17 19:48:50.390054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d341c889c0 pid: 81532 (default app) Aug 17 19:48:50.390426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafc10-0411-4586-ab5e-27ceb8f270d7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:50.390707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafc10-0411-4586-ab5e-27ceb8f270d7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:50.390976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafc10-0411-4586-ab5e-27ceb8f270d7 None None] PATH_INFO: `/v3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:50.395200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Aug 17 19:48:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:48:57.035008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:57.035622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:57.035622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:57.039955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:48:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:48:57.046718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:57.047248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:57.047248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:57.116482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: INFO dbcounter [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] Registered counter for database keystone Aug 17 19:48:57.136348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG oslo_db.sqlalchemy.engines [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 19:48:57.146389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writer thread running {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 19:48:57.336343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81533) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 19:48:57.336663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG passlib.handlers.bcrypt [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81533) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 19:48:57.394129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:48:57.395747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:48:57.534165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:48:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 17 19:48:58.029160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:48:58.102981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: INFO dbcounter [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] Registered counter for database keystone Aug 17 19:48:58.125629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG oslo_db.sqlalchemy.engines [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81532) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 17 19:48:58.132618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writer thread running {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 17 19:48:58.251710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01a72f-e3e9-4415-8d46-929093b42173 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:48:58.256504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:48:58.257145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:58.257421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:58.257703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:58.266764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:48:58.317330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 19:48:58.317579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60618a-8827-46be-bcea-83e3bf86ce5d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:48:58.328326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 17 19:48:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:48:58.334823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:48:58.349711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6511d-b416-4cba-abd0-0a4dc7603d1c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:48:58.355668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:48:58.356817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:48:58.357100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:48:58.357369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] PATH_INFO: `/v3/projects/79e7870848b34ea38cb8567a241b0ce4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:48:58.359157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] RBAC: Authorizing `identity:get_project(project_id=79e7870848b34ea38cb8567a241b0ce4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:48:58.411571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 17 19:48:58.411910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f063b-3326-4bb1-9d65-6ce332472f3a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:48:58.416668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Aug 17 19:48:58 2022] GET /identity/v3/projects/79e7870848b34ea38cb8567a241b0ce4 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:00.287286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c298348-c5da-46aa-acb4-c5f05be899bf None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:00.287854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c298348-c5da-46aa-acb4-c5f05be899bf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:00.287854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c298348-c5da-46aa-acb4-c5f05be899bf None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:00.294624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:00.303857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:00.304334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:00.304425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:00.333202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:00.334812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:00.344544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 17 19:49:00.608634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c298348-c5da-46aa-acb4-c5f05be899bf None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:00.621673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c298348-c5da-46aa-acb4-c5f05be899bf None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:00.626408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:00.627009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:00.627283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:00.627562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:00.635586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:00.637397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9413140-6250-47b1-8190-a7ac2ed43151 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:00.646244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 17 19:49:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:49:00.652224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:00.663288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded7472-fd28-4f16-8e11-f2d92847e57b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:00.668252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:00.669266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:00.669585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:00.669845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:00.671159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:00.673015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfbe10-a630-4e3f-a1d3-4efb6197e75b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:00.674600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Aug 17 19:49:00 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:02.673899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50706-59e7-402a-b7c5-807a17b86bdc None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:02.674911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50706-59e7-402a-b7c5-807a17b86bdc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:02.674911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af50706-59e7-402a-b7c5-807a17b86bdc None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:02.675819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:02.681088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:02.681501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:02.681654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:02.697566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38265f-261d-464e-a082-d86339c71299 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:02.697873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38265f-261d-464e-a082-d86339c71299 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:02.698182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38265f-261d-464e-a082-d86339c71299 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:02.699501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:02.700255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:02.701578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:02.704372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:02.704655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:02.704930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:02.748661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81532) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 17 19:49:02.749078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG passlib.handlers.bcrypt [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81532) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 17 19:49:02.792927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:02.794598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:02.796281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:02.905189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:03.060309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.084476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.171129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c370-73b1-4a03-880f-f75492e4c027 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.172642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.205890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.206530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.206807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.207071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.219205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:03.227691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:03.228628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:03.230165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:03.234839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.283879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7f91-9a8e-4161-bece-0c3cb1ba2dc2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.286762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.324248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.329182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.330107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.330387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.330655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.343113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeb3c3-ddce-4073-a842-8831498ec1c6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.377896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.378508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.378786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.379052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.379727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:03.380022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.381447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.393602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:03.398787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.399194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.400313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.498711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d05aa-035a-4539-b639-be18b37adef9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.504177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:49:03.509938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.512850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.535108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.535738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.536011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.536281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.537434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.538978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.540616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:03.546948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.564925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.618788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46261e-f104-49e5-9bbc-24c73e7acf92 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.653040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8572b-d810-4347-b973-9fbb0a7ab519 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.654812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.655711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.655983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.656257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.679442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.680933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.694927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.695557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.695831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.696102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.697220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.698782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:49:03.699677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:03.700660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:49:03.702430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:49:03.702430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:49:03.702430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:03.702638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:03.702716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:03.706657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.706811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:03.764739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.790559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:03.810241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53071015-1c89-4a66-87b8-414716817bdb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.810463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648453b0-a002-46d9-9df5-657f001e2259 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:03.845446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.846092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.846367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.846654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.846826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:03.847378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.domain.id=default, user.id=3a9bd68510704bb0a6eea1e00548849b) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:03.847521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:03.847648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:03.847772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.847911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:03.848559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:03.848856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:03.849114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea12ce-1965-4e1f-ae44-7dbef14a3791 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.850293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234eb3f-6afe-466b-acd3-7dee7b67cf36 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:03.858206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/role_assignments?user.id=3a9bd68510704bb0a6eea1e00548849b&scope.domain.id=default&role.id=c974a286ee1942c8bab82b232c6af142 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:03.874019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:06.082277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe250c0-859f-45fc-b256-1bdd990fff72 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.082852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe250c0-859f-45fc-b256-1bdd990fff72 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.083132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe250c0-859f-45fc-b256-1bdd990fff72 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.084407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:06.089969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20867bf2-3ee8-4b41-90c6-37f62cc3e0d3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.090920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20867bf2-3ee8-4b41-90c6-37f62cc3e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.091469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20867bf2-3ee8-4b41-90c6-37f62cc3e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.118618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20867bf2-3ee8-4b41-90c6-37f62cc3e0d3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:06.120248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20867bf2-3ee8-4b41-90c6-37f62cc3e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:06.195374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867824f0-f268-4d97-b7fa-920cbd77af85 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.195718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867824f0-f268-4d97-b7fa-920cbd77af85 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.195918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867824f0-f268-4d97-b7fa-920cbd77af85 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.197552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 12/23] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:06.202705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd51e3-1e17-4d08-8a1e-3eec5ca21d9a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.203209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd51e3-1e17-4d08-8a1e-3eec5ca21d9a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.203503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd51e3-1e17-4d08-8a1e-3eec5ca21d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.207949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 12/24] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:06.238602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-becd51e3-1e17-4d08-8a1e-3eec5ca21d9a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:06.239810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-becd51e3-1e17-4d08-8a1e-3eec5ca21d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:06.319559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 13/25] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:06.396832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0474f1-b61d-47d4-9683-91ba464b996b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.397475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0474f1-b61d-47d4-9683-91ba464b996b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.398027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0474f1-b61d-47d4-9683-91ba464b996b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.399613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:06.405114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.405399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.405711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.435958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:06.437189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:06.505653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:06.531993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:06.673984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0474f1-b61d-47d4-9683-91ba464b996b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:06.770015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:06.784699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:06.846381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0474f1-b61d-47d4-9683-91ba464b996b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:06.909754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:06.910395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.910669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.910937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.931248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5ab7c9a1-56da-4706-b5c9-70b14d56636b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:06.936126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:06.937113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:06.937900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:06.939111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8282-c5a6-41f4-9c69-935a560cd50d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:06.939635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b17e-05f9-45ba-bb70-549de10dc23a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.939909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b17e-05f9-45ba-bb70-549de10dc23a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.940177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431b17e-05f9-45ba-bb70-549de10dc23a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.941463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 15/29] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:06.947287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b17e-05f9-45ba-bb70-549de10dc23a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:06.981571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:06.982351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:06.982636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:06.982904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:06.994475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:06.995865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:49:07.012283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:07.013187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:49:07.018923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:49:07.018923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:49:07.018923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:07.018923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:07.018923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.062021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.064546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431b17e-05f9-45ba-bb70-549de10dc23a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.101107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.110261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.111736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.112441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.112898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.114099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.116677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.150153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcac1f-d881-4098-87fc-c09d33e7d555 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.190695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.191746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.192231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.192726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.193650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:07.194141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.195796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749084-8251-4eb6-bc41-35c2a8dd7f48 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.210041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:07.211855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.212349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.212805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.233207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:07.234797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:07.258816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 16/32] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 17 19:49:06 2022] POST /identity/v3/roles => generated 219 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:07.259840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.319111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 17/33] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:07.320353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.368280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.501476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22391f6c-4985-4a07-8913-67c46f02b843 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.517993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cd293-aa89-4edd-9815-5018333c270a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.558603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.559240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.559521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.559792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.560458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:07.560760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.562376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.591735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 17/34] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:07.598301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.600522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.601204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.601485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.601793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.605236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.606774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.608446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 18/35] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:07 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:07.616059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.674118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.771396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc67739-c2bc-454c-8ee7-4cae623ef09e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.806700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:07.812230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.813016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.813139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.813404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.814125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.815632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.820273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa18cc-4f2b-479c-b4a7-bbcd77662d9f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.854260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 18/36] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:49:07 2022] POST /identity/v3/projects => generated 301 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:07.855144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.863532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:07.864206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.864487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.864758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.866169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:07.868897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc966952-a152-402e-abe4-3736b2e6429b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:07.877479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:07.877479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f487-2deb-437d-8383-1eb053ca8e27 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.877479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f487-2deb-437d-8383-1eb053ca8e27 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.877479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f487-2deb-437d-8383-1eb053ca8e27 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.877479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:07.879181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82fa9-3c99-4676-bce7-b563ea36e5c8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:07.879482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82fa9-3c99-4676-bce7-b563ea36e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:07.879757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c82fa9-3c99-4676-bce7-b563ea36e5c8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:07.881176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 21/39] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:07.882477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c82fa9-3c99-4676-bce7-b563ea36e5c8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:07.987143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe49ae-daf0-4498-83c9-81ccbd1c0df3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:07.989106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:08.006963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c82fa9-3c99-4676-bce7-b563ea36e5c8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:08.027169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:08.027719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.027999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.028269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.028945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:08.030401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d937096-7606-416a-adc6-1ab1438a3883 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:08.046242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:08.047254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.047547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.047785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] PATH_INFO: `/v3/domains/default/users/3a9bd68510704bb0a6eea1e00548849b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.048632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 19/40] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 17 19:49:07 2022] POST /identity/v3/roles => generated 225 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:08.053704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3a9bd68510704bb0a6eea1e00548849b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:08.053925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefec87-37fd-47fb-be51-596314628c30 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.053925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefec87-37fd-47fb-be51-596314628c30 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.053925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefec87-37fd-47fb-be51-596314628c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.054137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:08.078348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aeefec87-37fd-47fb-be51-596314628c30 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:08.080015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aeefec87-37fd-47fb-be51-596314628c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:08.158468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 20/41] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:08.159787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.160075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.160346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.175854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Aug 17 19:49:07 2022] PUT /identity/v3/domains/default/users/3a9bd68510704bb0a6eea1e00548849b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:08.192885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:08.194645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:08.307270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:08.440654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:08.470121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:08.567291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:08.592639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:08.598644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05186bd4-e638-4ead-a51f-f8e6d7039e6e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:08.644593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:08.645257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.645560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.645849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.647276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:08.648958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:08.653664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 23/44] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:08.660294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:08.690114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66acd7-6975-4880-83fe-29f3b6211ff3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:08.729433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:08.730182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.730475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.730747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.731595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:08.733014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5469-36cf-4de6-a66d-ac12eda56fed None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:08.753985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:08.756304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 22/45] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 17 19:49:08 2022] POST /identity/v3/roles => generated 223 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:49:08.768100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17662f91-669a-4f11-a4c8-e35640c95078 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:08.804438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:08.805122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:08.805355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:08.805661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:08.806346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:08.807931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927144f1-a3dc-4c69-bb3d-bbd4d1ca1881 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:08.841397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:49:08 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:10.090520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d21c1-dbf1-41e7-ba9f-246ec17509d5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.091045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d21c1-dbf1-41e7-ba9f-246ec17509d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.091045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d21c1-dbf1-41e7-ba9f-246ec17509d5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.092403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:10.098146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5efc-6acb-4dd3-9627-edcf3a5c76ef None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.098514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5efc-6acb-4dd3-9627-edcf3a5c76ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.098705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5efc-6acb-4dd3-9627-edcf3a5c76ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.118244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-85da5efc-6acb-4dd3-9627-edcf3a5c76ef None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:10.119820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85da5efc-6acb-4dd3-9627-edcf3a5c76ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:10.196785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:10.458360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d21c1-dbf1-41e7-ba9f-246ec17509d5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:10.675165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:10.696784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d21c1-dbf1-41e7-ba9f-246ec17509d5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:10.734282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:10.734947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.735222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.735491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.736595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f3c1-dd37-4f9b-8737-4fe5d49510f6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.736888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f3c1-dd37-4f9b-8737-4fe5d49510f6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.737210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687f3c1-dd37-4f9b-8737-4fe5d49510f6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.738595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 26/49] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:10.743774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.744069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.744353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.747060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:10.751836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:10.752933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:10.753975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 24/50] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:10.759350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:10.762938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:10.764155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:10.837316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:10.842596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 27/51] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:10.868664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb79b8-e812-4cad-8ac9-31c0e4071fd6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:10.907346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:10.907967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:10.908245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:10.908515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:10.909191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:10.909492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:10.910922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:10.919987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:10.925683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:11.059600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf1525-abec-4ed4-891f-0a9abe671f23 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:11.094580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:11.095221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:11.095501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:11.095774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:11.096949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:11.098438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:11.100250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:10 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:11.105956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:11.108054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:11.110195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:11.179924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:11.212369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859f0cb-1f8f-4ae6-99ab-c1a24f04f626 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:11.213792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cee22-3ac4-4d23-9365-9c82b7f5fe0b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:11.245563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:11.246211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:11.246488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:11.246752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:11.247919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:11.248778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:11.249356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:11.249457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:11.249716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:11.249990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:11.250970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:11.251195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:11.252608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:11.257352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:11.257472 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:11.259506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:11.329570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:11.359568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa4f4c-6d67-4654-8f2d-42f7e7274752 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:11.361665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2b149-3140-4c66-ab39-86eb089eb779 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:11.394794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:11.395412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:11.395701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:11.395975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:11.396707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.domain.id=default, user.id=3a9bd68510704bb0a6eea1e00548849b) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:11.397027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:11.397122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:11.397777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:11.398063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:11.398335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:11.398537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3715f-1854-433b-a33c-e4641acb2dd6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:11.398985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:11.400419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05cf35-ecf9-4147-b040-acdfcf945b4a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:11.405951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:11.406069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 17 19:49:11 2022] GET /identity/v3/role_assignments?user.id=3a9bd68510704bb0a6eea1e00548849b&scope.domain.id=default&role.id=c974a286ee1942c8bab82b232c6af142 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:11.430804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 17 19:49:11 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:13.316327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08490b-3cfc-4890-85a6-07f33e267127 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:13.316881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08490b-3cfc-4890-85a6-07f33e267127 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:13.316881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08490b-3cfc-4890-85a6-07f33e267127 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:13.318328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:13.324155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:13.324655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:13.324655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:13.344445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:13.345706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:13.419487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:13.685490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08490b-3cfc-4890-85a6-07f33e267127 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:13.791036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08490b-3cfc-4890-85a6-07f33e267127 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:13.828447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:13.828993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:13.829271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:13.829513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:13.830754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:13.832276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3693dba-7be0-45a6-b588-01c82c982a2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:13.833887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:13.839803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:13.940822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13758a38-5203-4b37-9b76-e8fd8cebae16 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:13.962715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:13.982066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:13.982670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:13.982945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:13.983215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:13.983859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:13.985477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585813d-1a12-4ab1-ac57-b4e6f1c4178c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:14.013819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:13 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:16.858310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2934bd-519d-4103-bb07-650c4d98583b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:16.858310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2934bd-519d-4103-bb07-650c4d98583b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:16.858310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2934bd-519d-4103-bb07-650c4d98583b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:16.858310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:16.871101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442720-0b94-4bf3-aced-4c846f7c088d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:16.871958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442720-0b94-4bf3-aced-4c846f7c088d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:16.872448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54442720-0b94-4bf3-aced-4c846f7c088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:16.898687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-54442720-0b94-4bf3-aced-4c846f7c088d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:16.900247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-54442720-0b94-4bf3-aced-4c846f7c088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:16.999576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 33/63] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:17.051179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926b85b-50b8-4c34-85b2-a6eba7d1e970 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.051514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926b85b-50b8-4c34-85b2-a6eba7d1e970 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.051724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926b85b-50b8-4c34-85b2-a6eba7d1e970 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.053308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 31/64] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:17.058987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a2e35-30fc-416c-a795-f675ce21e317 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.059274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a2e35-30fc-416c-a795-f675ce21e317 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.059551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a2e35-30fc-416c-a795-f675ce21e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.082457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e91a2e35-30fc-416c-a795-f675ce21e317 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:17.083659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e91a2e35-30fc-416c-a795-f675ce21e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:17.172531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 34/65] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:17.358762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926b85b-50b8-4c34-85b2-a6eba7d1e970 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.363354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:17.390723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaafb31-3e6c-4eea-a3a2-b190102639bf None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.391056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaafb31-3e6c-4eea-a3a2-b190102639bf None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.391277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaafb31-3e6c-4eea-a3a2-b190102639bf None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.392873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 35/66] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:17.397838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.398120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.398393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.426961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:17.428085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:17.479371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:17.518073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926b85b-50b8-4c34-85b2-a6eba7d1e970 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:17.529384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 36/67] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:17.530644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.556264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:17.557180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.557661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.558073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.570093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:17.575629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:17.576767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:17.577756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:17.579725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 32/68] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:17.590718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.642092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12631e-30d9-4a1b-a01d-44af885041c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:17.671973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:17.680139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:17.680841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.681226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.681639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.693628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1d90ba4e-71d7-4461-b020-394e6bc34abe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:17.700962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:17.701911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:17.702666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 37/69] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:17.706892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76af3f-7e52-47aa-bb27-59d9921433d8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.706892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76af3f-7e52-47aa-bb27-59d9921433d8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.706892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76af3f-7e52-47aa-bb27-59d9921433d8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.706892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 38/70] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:17.711155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.711453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.711818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.726762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3711700-24ef-4bb0-b42c-b0ef5247cb72 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:17.730389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:17.731423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:17.749802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:17.767809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:17.768707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.769160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.769639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.770573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:17.771079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:17.772641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:17.782047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 33/71] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:17.783073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.802528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 39/72] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:17.803709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.886674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976ccf-1d3e-4ab8-99f2-0435001736dc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:17.909781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32e2de-d82c-4312-8afb-c295ecb71acd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:17.913606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:17.926089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:17.926702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.926975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.927242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.927908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:17.928217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:17.929647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ad6e0-cbca-444f-8f11-bbab371875ed None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 34/73] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3580e6-f19e-40e3-af2f-56e0705a1a29 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3580e6-f19e-40e3-af2f-56e0705a1a29 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3580e6-f19e-40e3-af2f-56e0705a1a29 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 35/74] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:17.942697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3580e6-f19e-40e3-af2f-56e0705a1a29 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.956375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:17.957022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:17.957299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:17.957610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:17.958745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:17.960184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:17.965639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 40/75] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:17.965639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:17.994107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.050826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3580e6-f19e-40e3-af2f-56e0705a1a29 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.082177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3acec1-ddbc-482e-be7e-675d0e02c996 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.086496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.087126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.087404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.087666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.098701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3b281ba1-7373-4d1e-90b0-eee7cab3c204 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:18.103347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:18.104334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:18.105092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 36/76] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:18.106249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.106249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.106391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.117587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.118254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.118538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.118812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.123665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:18.125065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:18.133828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.135304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.136990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 41/77] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:17 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:18.138404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.172701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.196267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 37/78] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:18.197123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.223616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.245958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee99c6-4834-4472-ac16-6aaf1bc4022c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.283019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.283643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.283919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.284187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.303251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462ac06-a0aa-4f8a-ab61-aa6b163f3054 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.342865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.343622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.343960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.344233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.345358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.346951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.348730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 38/79] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:18.349288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.361589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.363529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.401121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.449485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.461041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6117fd3-d26a-44c9-87b0-81ba4d2d5f1e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.469299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 42/80] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:18.470183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.498710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.499320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.499603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.499861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.500513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:18.500826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.502268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.510630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 39/81] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:18.512714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.623510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b034c67-97f6-4f1d-8d1a-479d276bbc72 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.632815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.659255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.659868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.660143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.660413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.661068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.662653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.679256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236aecf-6d4b-418f-bb60-e849484ae177 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.693148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 40/82] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:18 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:18.694006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.729753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.730734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.731012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.731305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.796524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.818738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3d5ab-1fb2-48cf-85e6-27b4988b1fda None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:18.820524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:18.822064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:18.864239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:18.864660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:18.865498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:18.865498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:18.881016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:18.885915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:18.887356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 41/83] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:18.888170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:18.921832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:18.952203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 43/84] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:18.953955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.026891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3c30a-85eb-45ca-bc41-8ca9fa537e18 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.047815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.072915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b705d6-57a6-4fc5-a20b-5985c1f44885 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.073831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.074642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.075069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.075461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.076804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.078790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.080655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 42/85] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.081729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.121274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.122108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.122397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.122680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.124167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.126340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.133627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 44/86] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:18 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.133627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.198717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.209391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cca0c-3946-42fa-af94-c002f01886f8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.253092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.253814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.254355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.254638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.255392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:19.256115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.257818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.265121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d4bff-7e26-4924-ab62-207d8ad75583 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.273634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 43/87] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.277383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.305325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.306408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.306882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.307447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.309671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.311613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.319098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 45/88] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.320277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.340309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.401860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e17a2-fd79-41c4-8bb2-54b674b311a1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.438053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.439019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.439517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.439975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.440918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=3a9bd68510704bb0a6eea1e00548849b) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:19.441450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.443282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.446329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a598dc-f0c2-4a2c-8319-9b76623e39ae None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.449397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 44/89] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/role_assignments?user.id=3a9bd68510704bb0a6eea1e00548849b&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.450625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.478013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.491705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.492357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.492644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.492977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.626811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.628793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.644905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0c701-7a7c-4b35-aab3-79bfe5557d44 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.661922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.738296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.739335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.739833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.740298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.746266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.747029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 46/90] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:19.747691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.748757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.751220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 45/91] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.757169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.856677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3db7e-2bbd-4ef7-aa5b-aa4e1112e036 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:19.885561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.892983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:19.893619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:19.893908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:19.894183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:19.894934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=373336f834404161825914878c086dbf) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:19.895253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:19.901570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:19.903029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:19.903851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:19.941285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:19.963834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877090e-ebca-41b6-930d-59cb5805dfcc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:20.019262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:20.019914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:20.020416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:20.020696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:20.024579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:20.027158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:20.029447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 46/93] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:20.035923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:20.158804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e0f1-0812-4805-8971-b5f330feda0f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:20.210196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:20.211642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:20.212123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:20.212616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:20.247057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7332f-744b-471c-8946-93e0dfbcdb0e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:20.291723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:20.315338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:20.315964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:20.316243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:20.316512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:20.317252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=373336f834404161825914878c086dbf) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:20.317571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:20.321421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:20.327886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:20 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:20.331791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:20.333306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:20.484866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 48/95] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:20.490480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:20.533925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:20.645206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705c396-294c-45cc-9802-adf1d588a477 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:20.686283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:20.686947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:20.687264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:20.687543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] PATH_INFO: `/v3/projects/fb9d009f81ea4669b007ed5ce8203f01` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:20.688751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] RBAC: Authorizing `identity:get_project(project_id=fb9d009f81ea4669b007ed5ce8203f01)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:20.690265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0b3af-2259-4e35-8029-b386dee05655 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:20.692173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:20 2022] GET /identity/v3/projects/fb9d009f81ea4669b007ed5ce8203f01 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:20.698674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:20.795400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:20.818777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697605fd-f49e-4a1c-9c1c-427920a58335 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:20.866265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:20.866894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:20.867178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:20.867456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:20.868203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=fb9d009f81ea4669b007ed5ce8203f01, user.id=373336f834404161825914878c086dbf) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:20.868518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:20.870099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c99541-a1d0-49dc-9c0d-1f10df672347 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:20.876015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:20 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=fb9d009f81ea4669b007ed5ce8203f01&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:21.263764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a54962-5f50-480d-b557-59973153e32c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:21.264285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a54962-5f50-480d-b557-59973153e32c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:21.264285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a54962-5f50-480d-b557-59973153e32c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:21.265969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:21.269827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a88d-925f-4c15-88c5-942b76a8d896 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:21.270234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a88d-925f-4c15-88c5-942b76a8d896 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:21.270429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8a88d-925f-4c15-88c5-942b76a8d896 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:21.271740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 50/99] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:21.274678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:21.274963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:21.275238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:21.297368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:21.297686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:21.297947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:21.303419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:21.304623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:21.326619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:21.328128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:21.419236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 50/100] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:21.441283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 51/101] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:21.710579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:21.729209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:21.816659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cab88e-445e-4dc5-b4b2-f6e5cba0e2ee None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:21.854113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:21.854773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:21.855086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:21.855355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:21.866067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:21.870805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:21.871629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:21.872398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 51/102] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:21.878036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:21.991122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.024366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.029970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fad21-6fde-4d3c-b55d-884b79f879c0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.085590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.086604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.087070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.087534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.088641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:22.089133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.090834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.105060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 52/103] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:22.122997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.169878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.235820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ad0f-111b-45c8-aace-23c744c06988 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.293375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e862b-7539-42d6-9224-6dba0e01feb6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.309567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.310837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.311302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.311756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.314238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.315972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b75d3f-59ed-4e40-a182-cd64e2299c21 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.317832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 52/104] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:22.323632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4196-5d72-45fa-b832-e6fc0475e56a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.326212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4196-5d72-45fa-b832-e6fc0475e56a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.326701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d4196-5d72-45fa-b832-e6fc0475e56a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.330762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 53/105] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:22.331668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4196-5d72-45fa-b832-e6fc0475e56a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.370126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.371300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.373145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.374320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] PATH_INFO: `/v3/users/2d53e964d031458bb42cf64fb45eb115` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.397940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.402694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] RBAC: Authorizing `identity:get_user(user_id=2d53e964d031458bb42cf64fb45eb115)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.404419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c873d-af7b-42f1-9406-94573f8219b8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.409734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 53/106] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:22 2022] GET /identity/v3/users/2d53e964d031458bb42cf64fb45eb115 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:22.411516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.411880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.412245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.444566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:22.448465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:22.509192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d4196-5d72-45fa-b832-e6fc0475e56a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.568111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.568731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.569003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.569264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.570544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.572018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04f8c6-c28b-4be7-bc13-5fc0e525ae3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 54/107] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:22.589676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.608551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.627569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 54/108] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:49:22 2022] POST /identity/v3/projects => generated 302 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:22.724190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fff9b6-5c2b-4185-8b94-7841a98a4a65 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:22.749940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:22.761060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:22.761714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.762001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.762275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.834375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:22.835853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:22.879215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733145-3d38-4f0f-a38b-3640164fcf55 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.879838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733145-3d38-4f0f-a38b-3640164fcf55 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.880301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733145-3d38-4f0f-a38b-3640164fcf55 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.881896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 55/109] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:22.889628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:22.890486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:22.890987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:22.930525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:22.932121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:22.940149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 55/110] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:22.945996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:22.982487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:23.035752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 56/111] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:23.040491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.098202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebe38e-4e16-43bc-8422-63659a981d65 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.135671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.136231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.136506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.136774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.138062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:23.139577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:23.141576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 56/112] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:22 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:23.147707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.166297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3143cb-a90b-428b-987c-79dfa2bd4b11 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.205078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:23.216493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.217179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.217634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.218115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.233620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1fa189a7-7f43-4ec5-97a3-38c9048e39e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:23.240892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:23.241786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:23.242570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:23.245767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 57/113] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:23.247688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc8ab-4657-4055-92b8-b26aa315bbb2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.248183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc8ab-4657-4055-92b8-b26aa315bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.248647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc8ab-4657-4055-92b8-b26aa315bbb2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.250790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 58/114] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:23.254848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc8ab-4657-4055-92b8-b26aa315bbb2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.271125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:23.282155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc527d85-fd4a-448b-9e01-f640142bb51a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.346870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.347769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.348403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.348697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.349506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=2d53e964d031458bb42cf64fb45eb115) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:23.349891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:23.351459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905d701-908b-403c-bc13-66d6a342fb1d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:23.358556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 57/115] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/role_assignments?user.id=2d53e964d031458bb42cf64fb45eb115&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:23.359984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.360416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.360700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.391250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:23.393795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:23.509222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 58/116] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:23.514961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc8ab-4657-4055-92b8-b26aa315bbb2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.570051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.570869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.570941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.571195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.571875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:23.572169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:23.573716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:23.583359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 59/117] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:23.592622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.601583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.727174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:23.773663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:23.805884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a49ae-3556-48ad-bac7-087797b70eb9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.845107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc90d-7d76-42b5-8425-e45940af330d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:23.849324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.850262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.850782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.851261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.852747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:23.854582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65724544-5b95-41f0-ad60-b244381addb1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:23.856874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:23.858502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ee6a-301c-42ce-a339-e48ce81c7fb1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.859034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ee6a-301c-42ce-a339-e48ce81c7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.859627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635ee6a-301c-42ce-a339-e48ce81c7fb1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.861072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 60/119] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:23.864912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ee6a-301c-42ce-a339-e48ce81c7fb1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:23.898547 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:23.898952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.899214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.899488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.919200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3bec6803-17cf-43a7-a605-eaf9ea5d3d4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:23.924379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:23.925251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:23.926053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:23.934012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 60/120] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:23.935380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:23.935668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:23.935944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:23.971355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:23.972604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:23.979739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635ee6a-301c-42ce-a339-e48ce81c7fb1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.050029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.050686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.050966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.051242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.054164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.055657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.057431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 61/121] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:23 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:24.063276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.075315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 61/122] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:24.076198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.111639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.160756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.254599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f346fa-75e5-4f27-b029-723c72ec546c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.268029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd95295-878b-4732-9d4a-57496fe47162 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.320368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.321229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.321505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.321809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.322489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:24.322849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.324236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.330865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.331652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.331930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.332202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.343058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 62/123] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:24.346158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.353441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:24.363276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:24.364155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:24.364920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:24.367244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 62/124] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:24.369382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.464550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.484213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea243b-c94e-4bf7-8406-be66371401eb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.494780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.499198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22299e9-a807-4a2d-8448-f82b222078ff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.530820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.531506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.531789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.532210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/users/3a9bd68510704bb0a6eea1e00548849b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.534950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, user_id=3a9bd68510704bb0a6eea1e00548849b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.536772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.544654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.545347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.545658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.545877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.549672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.551212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.553084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 63/125] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:24.555615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.557481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 63/126] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:24 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/users/3a9bd68510704bb0a6eea1e00548849b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:24.558473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.699025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ef06c-50e1-421b-8d6a-33ffc885a388 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.742821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.743511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.744368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.744680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.765789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:24.771554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:24.772461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:24.773244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 64/127] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:24.773913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.777796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.796157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f13219-6b7e-410b-9b92-58c7b7c8f67a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.835373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.835991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.836366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.836637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.837419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:24.837742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.839188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.849053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:24.849937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.882243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:24.885441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bce6eb-0989-417e-b4de-d2a4f31968e1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:24.925503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:24.926306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:24.926659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:24.926940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:24.928388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:24.929918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:24.931876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 65/129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:24.932774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:24.964548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114680-cba7-4513-8f74-39ffd622bbff None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.002016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.002726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.003007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.003275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.004106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:25.004407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.005826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.007587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.016306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 65/130] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:25.016969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.039849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1312f-7b42-4d2b-84c6-6fabaa805b3d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.077657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.078304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.078584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.078857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.080220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.081733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.083720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 66/131] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:24 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:25.084630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.138915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.149356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64f925-775e-422b-91e2-9e3621d053fc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.185566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.186449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.186791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.187150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/users/373336f834404161825914878c086dbf/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.189444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, user_id=373336f834404161825914878c086dbf, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.190972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.212070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 66/132] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:25 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/users/373336f834404161825914878c086dbf/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:25.213018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.258834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda4dc-3098-47d1-8c92-542c543c8cbd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.300676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.301863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.302344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.302875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.304255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.306182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e74a7-3265-453f-ad15-68b703a010e1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.308095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 67/133] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:25 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:25.309623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d47c-dcf2-4a57-bfa1-b325e1763fa5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.310127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d47c-dcf2-4a57-bfa1-b325e1763fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.310625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161d47c-dcf2-4a57-bfa1-b325e1763fa5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.312284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 68/134] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:25.318218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.318817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.319291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.340949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:25.342721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:25.349953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.356855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f308bb5-7ded-46f5-b853-61767321fae9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.402236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.403291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.403762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.404250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.406071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.408558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.411384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:25 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:25.412849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.438605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.492476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 69/136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:25.493169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.585476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479b50-320d-4028-91fe-3fbdc4c902d9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.636932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.637601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.637882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.638148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] PATH_INFO: `/v3/projects/fb9d009f81ea4669b007ed5ce8203f01` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.641987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] RBAC: Authorizing `identity:get_project(project_id=fb9d009f81ea4669b007ed5ce8203f01)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.645895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.649367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 68/137] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:25 2022] GET /identity/v3/projects/fb9d009f81ea4669b007ed5ce8203f01 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:25.654818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.698940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b6b80-3133-44bc-96d6-5239ca0d4bea None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.767412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.768097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.768375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.768653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/users/373336f834404161825914878c086dbf/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.773693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, user_id=373336f834404161825914878c086dbf, role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.775863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.797886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.806131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 70/138] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:25 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/users/373336f834404161825914878c086dbf/roles/1f37e1386b624e268925de2a48d79cc1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:25.807051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:25.813027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:25.834550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e749a-99a1-4980-b98a-03d59952964a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:25.904331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:25.904928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:25.905208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:25.905482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] PATH_INFO: `/v3/projects/fb9d009f81ea4669b007ed5ce8203f01/users/373336f834404161825914878c086dbf/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:25.909203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb9d009f81ea4669b007ed5ce8203f01, user_id=373336f834404161825914878c086dbf, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:25.911520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:25.935817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 69/139] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:25 2022] PUT /identity/v3/projects/fb9d009f81ea4669b007ed5ce8203f01/users/373336f834404161825914878c086dbf/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:49:26.029369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc034ae-f434-4fe4-8a57-e5846ad9ce55 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:26.094172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:26.094910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:26.095197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:26.095523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:26.096814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:26.098404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c98b5f-52ab-4885-a8fb-12c62d9a94cf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:26.100176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 71/140] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:26.106189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:26.114090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12b9c-b406-4bbb-8532-b31045408f22 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:26.114411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12b9c-b406-4bbb-8532-b31045408f22 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:26.114654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f12b9c-b406-4bbb-8532-b31045408f22 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:26.116082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 72/141] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:26.122794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:26.123115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:26.123346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:26.143459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:26.144698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:26.212223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:26.222174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 73/142] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:26.224233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea62f5-5d0e-4b67-beb1-d77f6cda8870 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:26.272334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:26.272957 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:26.273234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:26.273508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:26.274199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:26.275677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:26.300630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 70/143] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:26 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:26.689716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:26.807847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12cc49-382e-4630-bede-637b302cfe79 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:26.844973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:26.845908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:26.846332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:26.846769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:26.858599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:26.863609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:26.864666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:26.865637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:26.867571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 74/144] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:26.873673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:26.891481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:26.998696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5908c31-8146-46b1-ac37-ffa94cad995f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:27.044328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:27.045298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.045829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.046299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.047127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:27.047573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:27.049117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:27.061192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 71/145] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:27.066930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:27.135612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:27.183981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a9f04-dfd5-49f4-9acd-49a335e4f086 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:27.220672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:27.221649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.222139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.222560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] PATH_INFO: `/v3/users/2d53e964d031458bb42cf64fb45eb115` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.223864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] RBAC: Authorizing `identity:get_user(user_id=2d53e964d031458bb42cf64fb45eb115)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:27.225474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:27.227262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 75/146] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:27 2022] GET /identity/v3/users/2d53e964d031458bb42cf64fb45eb115 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:27.233057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:27.309437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:27.352852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26722d-0486-4a8d-87a7-24ba40957ee6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:27.392744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:27.393317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.393631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.393899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.395065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:27.396541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d66137-4697-4df3-8e55-c3f30da20359 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:27.398230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 72/147] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:27 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:27.404080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:27.517404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ea63f-641a-4969-8e30-4e36b1478e72 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:27.557414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:27.574606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:27.578339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.578623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.578899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/users/2d53e964d031458bb42cf64fb45eb115/roles/bdb8bd48f0394d9bb06f750023d9239f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.581079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, user_id=2d53e964d031458bb42cf64fb45eb115, role_id=bdb8bd48f0394d9bb06f750023d9239f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:27.582569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7490996-cac4-49e3-a3e6-4834efccfcf7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:27.612157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 76/148] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:27 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/users/2d53e964d031458bb42cf64fb45eb115/roles/bdb8bd48f0394d9bb06f750023d9239f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:27.828102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586c75-ee88-4386-8dd2-a4f2513ba4d6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.828279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586c75-ee88-4386-8dd2-a4f2513ba4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.828553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca586c75-ee88-4386-8dd2-a4f2513ba4d6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.830218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 73/149] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:27.836378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb21a0-c681-4096-bb9a-b460013bbcc8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:27.836885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb21a0-c681-4096-bb9a-b460013bbcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:27.836967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb21a0-c681-4096-bb9a-b460013bbcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:27.859071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eacb21a0-c681-4096-bb9a-b460013bbcc8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:27.860464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eacb21a0-c681-4096-bb9a-b460013bbcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:27.938691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 77/150] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:28.119133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178723de-e98f-40bd-b1cf-919f7d9f1c43 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:28.120177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178723de-e98f-40bd-b1cf-919f7d9f1c43 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:28.120652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178723de-e98f-40bd-b1cf-919f7d9f1c43 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:28.122568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 74/151] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:28.127827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:28.128114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:28.128388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:28.151492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:28.152754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:28.229572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 78/152] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:28.317119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178723de-e98f-40bd-b1cf-919f7d9f1c43 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:28.438032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178723de-e98f-40bd-b1cf-919f7d9f1c43 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:28.476076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:28.482574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:28.483794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:28.484466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:28.485000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:28.506033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:28.518230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:28.519202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:28.519994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 75/153] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:28.526765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:28.606030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:28.679628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a83dc9-6181-4abd-b9e1-e351304c4177 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:28.723996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:28.724545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:28.724820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:28.725091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:28.725799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:28.726166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:28.728018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:28.742346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 79/154] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:28.748414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:28.859182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:28.873563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e7226-95fd-47f6-80ca-348d60507600 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:28.901820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:28.902858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:28.903322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:28.903784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] PATH_INFO: `/v3/users/3a9bd68510704bb0a6eea1e00548849b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:28.905183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] RBAC: Authorizing `identity:get_user(user_id=3a9bd68510704bb0a6eea1e00548849b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:28.906875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:28.908975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 76/155] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:28 2022] GET /identity/v3/users/3a9bd68510704bb0a6eea1e00548849b => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:28.914706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:28.978037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:29.019991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be770e-af00-48e9-996b-9ba38bcb59cb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:29.054845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fcff5-1368-46fc-84e2-c18f89edbdc4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:29.056185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:29.057129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.057626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.058077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.069985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:29.125653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826419ec-2243-4348-8356-fb81266d247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:29.126692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826419ec-2243-4348-8356-fb81266d247b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.127242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826419ec-2243-4348-8356-fb81266d247b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.127697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826419ec-2243-4348-8356-fb81266d247b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.150484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826419ec-2243-4348-8356-fb81266d247b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-826419ec-2243-4348-8356-fb81266d247b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:29.161042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:29.162167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:29.163041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:29.170879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 80/156] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:29.175340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:29.177551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:29.178538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826419ec-2243-4348-8356-fb81266d247b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:29.267492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:29.319771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 77/157] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:49:29.326601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:29.392716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826419ec-2243-4348-8356-fb81266d247b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:29.463366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd213e-af1d-4dbd-a1cc-972e8f5063f9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:29.504447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:29.505132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.505415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.505717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.506396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:29.506695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:29.508082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:29.518607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:29.519597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.520059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.520520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.523891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:29.525717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457aea9-6be2-47cb-b7c7-da87d1bd9f96 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:29.527968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 78/158] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:29 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:29.529259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 81/159] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:29.529647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b76da2-285f-4127-b2a1-93853a662b07 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.530139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b76da2-285f-4127-b2a1-93853a662b07 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.530662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b76da2-285f-4127-b2a1-93853a662b07 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.532476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 79/160] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:29.538865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:29.539044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4bbb-7a23-4f04-886b-3c3334bf459b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.539280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4bbb-7a23-4f04-886b-3c3334bf459b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.539546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d4bbb-7a23-4f04-886b-3c3334bf459b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.540986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 80/161] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:29.542009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.542233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.542493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.566403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:29.568648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:29.613911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:29.660215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:29.697986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 81/162] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:29.699182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:29.797113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876e6be-8b44-4e84-ae91-03b412804681 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:29.854530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:29.855602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.856094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.856565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.860786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:29.862937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fc3132-458b-492f-b8b4-4a5d1eca66db None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:29.864803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:29 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:29.866308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:29.866792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:29.867260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:29.903530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:29.905619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:29.940819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6298e0-d197-4747-a0b1-ba68cb36d459 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.004527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.005169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.005466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.005754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.006503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=3a9bd68510704bb0a6eea1e00548849b) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:30.006802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:30.008336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:30.019885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 82/164] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:29 2022] GET /identity/v3/role_assignments?user.id=3a9bd68510704bb0a6eea1e00548849b&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:30.020929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.042195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 83/165] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:30.061654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:30.078248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.134074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:30.173943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092bf9e-5c2e-41f7-9620-8da140551866 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.214697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.215334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.215591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.215834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.273609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff2793-f00a-40e8-a6b5-acf35578c304 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.293026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:30.293026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:30.318218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:30.335066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.336211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.336835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.337304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.349137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:30.354895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:30.355839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:30.356874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:30.359360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 84/166] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:30.365107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.458085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 83/167] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:30.464049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.519991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:30.560485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e797-f333-418a-8a58-d2bb4902eea7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.655340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484bf2ce-6b6b-458e-a099-6f1a291b03d2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.665392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.665392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.665392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.665392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.665832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:30.665914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:30.667366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:30.687820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 85/168] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:30.689147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.693559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:30.697901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.698332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.698728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.699066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.700424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:30.702070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:30.703855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 84/169] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:30.708730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.858349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a07e-4312-43ae-96a4-94907b41ad96 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.887636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ca9c8-21e3-4d93-b1aa-172b0f293b48 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:30.923425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.924072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.924337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.924610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.940760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:30.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:30.948109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:30.948918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 86/170] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:30.950367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.952284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:30.952928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:30.953280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:30.953566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:30.954734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:30.956201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:30.958228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 85/171] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:30.959333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:30.979640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.065630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.134376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c767e8-2c1c-4580-9447-b4d45d82d8db None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:31.146529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d725-1cbd-4ebb-b341-4fdf783dc60c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:31.200004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:31.200631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.200904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.201169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.201886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:31.202183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:31.204857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:31.209053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:31.209721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.209998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.210267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.211021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=373336f834404161825914878c086dbf) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:31.211330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:31.214621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:31.224468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 87/172] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:31.231727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 86/173] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:31.232708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:31.238382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:31.355822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.404716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572cda-92c1-4a0d-a72a-e8d88b3baa2e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:31.485056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:31.486649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.487200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.490888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.510016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077b2e4-e53d-4f95-97fc-3bd10087d428 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:31.564287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.601721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:31.602686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.603128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.603657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] PATH_INFO: `/v3/users/373336f834404161825914878c086dbf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.606069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=373336f834404161825914878c086dbf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:31.611320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3aede3-2623-42f5-9d2f-29b1ab53e4b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:31.614000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 87/174] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:31 2022] GET /identity/v3/users/373336f834404161825914878c086dbf => generated 275 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:31.617003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:31.622077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92fbe8-d899-426c-8166-d75c0a1cfb3e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:31.622299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5257-3566-4c6d-8757-9aef4fe8bcd7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.622299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5257-3566-4c6d-8757-9aef4fe8bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.622548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad5257-3566-4c6d-8757-9aef4fe8bcd7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.624257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 88/175] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:31.625418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ccdb-e5aa-4377-814d-6ccb04301acf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.625729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ccdb-e5aa-4377-814d-6ccb04301acf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.625996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94ccdb-e5aa-4377-814d-6ccb04301acf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.627466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 89/176] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:31.628366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaea7b-f5eb-47b5-a305-22bb50f29d11 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.628637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaea7b-f5eb-47b5-a305-22bb50f29d11 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.628900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaea7b-f5eb-47b5-a305-22bb50f29d11 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.637679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 90/177] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:31.637679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaea7b-f5eb-47b5-a305-22bb50f29d11 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:31.723051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.776486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 88/178] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:31.778125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c814c5-0353-4447-95b3-e112fc31f7df None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.778406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c814c5-0353-4447-95b3-e112fc31f7df None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.778679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c814c5-0353-4447-95b3-e112fc31f7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.800232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eaea7b-f5eb-47b5-a305-22bb50f29d11 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:31.800661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e2c814c5-0353-4447-95b3-e112fc31f7df None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:31.802175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e2c814c5-0353-4447-95b3-e112fc31f7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:31.841198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:31.841864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.842139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.842517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.882352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 89/179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:31.895002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649f9e-3be8-4de9-88bf-3c2e3587a5c2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:31.895256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649f9e-3be8-4de9-88bf-3c2e3587a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:31.895740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20649f9e-3be8-4de9-88bf-3c2e3587a5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:31.908710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:31.920195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-20649f9e-3be8-4de9-88bf-3c2e3587a5c2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:31.921396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-20649f9e-3be8-4de9-88bf-3c2e3587a5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:31.926051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:31.927914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.006270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 90/180] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:32.007692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eeee8-e810-4294-9ecb-6dc3f4216d7b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.007978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eeee8-e810-4294-9ecb-6dc3f4216d7b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.008389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eeee8-e810-4294-9ecb-6dc3f4216d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.034796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e58eeee8-e810-4294-9ecb-6dc3f4216d7b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:32.036262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e58eeee8-e810-4294-9ecb-6dc3f4216d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:32.058905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 91/181] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:32.060222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.076591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.144965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 91/182] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:32.149851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286b60-c2d2-40fb-8abb-9bcf73f533df None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.149851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286b60-c2d2-40fb-8abb-9bcf73f533df None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.149851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4286b60-c2d2-40fb-8abb-9bcf73f533df None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.149851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 92/183] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:32.150323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286b60-c2d2-40fb-8abb-9bcf73f533df None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.193168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.214803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a136b2-07e2-4a0e-b983-6772478bb10a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.255642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.256389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.256539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.256819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] PATH_INFO: `/v3/projects/fb9d009f81ea4669b007ed5ce8203f01` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.257948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] RBAC: Authorizing `identity:get_project(project_id=fb9d009f81ea4669b007ed5ce8203f01)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:32.259473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084ff80-55bc-4d16-8fcf-1f0499bffc59 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.261936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 92/184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/projects/fb9d009f81ea4669b007ed5ce8203f01 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:32.266773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.266773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.266773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.286040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:32.287662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:32.300660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4286b60-c2d2-40fb-8abb-9bcf73f533df None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.338437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.339085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.339382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.339649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.340826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:32.342320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.343981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 93/185] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:32.344736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.377562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 93/186] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:32.377562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.385941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.471627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96977762-8f99-4bd7-9862-dc966a073788 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.483196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.502655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bb0d0-c78a-4dd3-9897-dbd16302a2ed None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.513520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.514286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.514564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.514902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.526918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:32.533392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:32.534332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:32.535143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:32.540882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 94/187] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:32.542512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.546992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.547612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.547880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.548143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.548878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=fb9d009f81ea4669b007ed5ce8203f01, user.id=373336f834404161825914878c086dbf) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:32.549173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:32.551149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.559290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 94/188] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=fb9d009f81ea4669b007ed5ce8203f01&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:32.562266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.711445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.724207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847862d-1503-417b-86fd-067cdb31d93a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.773876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.774439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.774742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.775007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.782151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6baec8-eeef-4755-a9c8-75586958a0eb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.788247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:32.793349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:32.794713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:32.795507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 95/189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:32.796765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.828646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.829447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.829836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.830189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.831231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=373336f834404161825914878c086dbf) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:32.831626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:32.833463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.845581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 95/190] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/role_assignments?user.id=373336f834404161825914878c086dbf&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:32.845581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.862815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:32.917698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472310a-6818-4cf8-bdc9-669d5faf2823 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.956132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:32.956796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:32.957067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:32.957314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:32.958068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:32.958370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:32.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:32.963448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f19f6-9d66-4e5e-8463-8628ed570553 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:32.969197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 96/191] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:32.970419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:32.990641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.011665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.012662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.012878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.013378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.014433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.016591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.023601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 96/192] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.024743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.116163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f793-0fb5-41bc-85db-1f3e0e1664de None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.148118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.161883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.162603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.163019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.163249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.175287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:33.176864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bbd0d-4b47-4ec5-a1ca-2cde49439b46 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:33.180539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:33.181469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:33.182370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 97/193] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:33.183026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.219003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.219714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.219976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.220297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.221057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:33.221412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.223042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.235337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 97/194] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.237690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.252142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.289171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a233-c9ee-49f1-b6e4-c129100a891e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.330162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.330796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.331071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.331336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.347322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.348777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.350458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 98/195] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.351218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.370368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45b4c-88ac-4205-b31f-576e3606a517 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.410839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.411809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.412112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.412435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.413207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.415013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.436225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.441900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 98/196] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:33 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:33.442791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.479121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52dca-7637-4ce4-b72a-6c169867f451 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.523453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.530858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.531487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.531762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.532029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.533332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:33.533667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.535096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.547761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 99/197] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.548599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.586063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fed4b-7158-47b0-870f-33ad5629793b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.644624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.645515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.645982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.646413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] PATH_INFO: `/v3/users/2d53e964d031458bb42cf64fb45eb115` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.648130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] RBAC: Authorizing `identity:get_user(user_id=2d53e964d031458bb42cf64fb45eb115)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.650200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.652406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 99/198] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/users/2d53e964d031458bb42cf64fb45eb115 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.653387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.705158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d707b-50e6-4ad9-9921-70740aa8f1f0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.758288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.759466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.759905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.760293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.761397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.808611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2d49c-450e-49dd-8e9a-18c2bc504fa1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:33.838611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:33.850099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.851572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.853378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:33.854169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:33.854446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:33.854709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:33.855851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:33.857679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:33.859608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 100/199] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:33.860399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.971803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 100/200] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:33.971803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:33.978313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7e280-ae85-4d73-a71b-0023fc099901 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.015935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.016559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.016830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.017091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.021938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.084230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.097766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.098218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9649a-daf2-49d4-8e8c-2c30c5439225 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.099278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.141558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.143547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.144015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.144470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.236244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 101/201] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:34.236244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.236530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.236530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.274775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.344960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827b963-dbc4-478e-b66e-f64de078a2a5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.370087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.374881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 101/202] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:34.376702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.385959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.386672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.386950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.387217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.389084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.390133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.392133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 102/203] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:34.393483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.504214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4c53b-9e43-467e-9c90-c656876a7027 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.506404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9840d-7345-4802-baa5-bbe49ad8ffb9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.507572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.546510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.547284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.547548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.547931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.549133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.550708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.552049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.552951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.553054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 103/204] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:34.553479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.553938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.554154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.555518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.557489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.560467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 102/205] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:34.561396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.639623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.681123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e80c6-5933-4e57-8259-f7c31e88e516 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.687597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf1a1c-27f0-4cf9-ae78-a1ad681e403b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.723683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.724309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.724585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.724862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.725641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=b99ce6a157914c46a719b50446253f89) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:34.725952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.726581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.727191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.727464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.727647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.727755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.728444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=b99ce6a157914c46a719b50446253f89) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:34.728809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.730598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85ff35-f40d-4560-94c4-667f3e21b12a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.735436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 103/206] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/role_assignments?user.id=b99ce6a157914c46a719b50446253f89&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:34.736459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:34.737067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 104/207] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/role_assignments?user.id=b99ce6a157914c46a719b50446253f89&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:34.893274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea10a99-5337-4e44-b597-d238f815be36 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:34.930327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:34.938920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:34.939814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:34.940336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:34.940611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:34.941365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, scope.project.id=0507e9f171104361a7ff74d326317861, user.id=2d53e964d031458bb42cf64fb45eb115) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:34.941707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:34.943204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a64355-e780-4b45-9c08-af3dc733ea97 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:34.950126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 104/208] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:34 2022] GET /identity/v3/role_assignments?user.id=2d53e964d031458bb42cf64fb45eb115&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:35.634260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6966-2f58-4d19-9720-796f60160231 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:35.634766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6966-2f58-4d19-9720-796f60160231 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:35.634766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f6966-2f58-4d19-9720-796f60160231 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:35.636319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 105/209] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:35.642317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955c9d-170b-4234-8842-f3dc473c98d7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:35.644254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955c9d-170b-4234-8842-f3dc473c98d7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:35.644254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82955c9d-170b-4234-8842-f3dc473c98d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:35.661905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-82955c9d-170b-4234-8842-f3dc473c98d7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:35.663197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-82955c9d-170b-4234-8842-f3dc473c98d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:35.731956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 105/210] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:35.824294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511362be-b287-40b0-88b8-721a2d862b34 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:35.824467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511362be-b287-40b0-88b8-721a2d862b34 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:35.824743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511362be-b287-40b0-88b8-721a2d862b34 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:35.826513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 106/211] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:35.834789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:35.835222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:35.835504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:35.855358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:35.856579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:35.928893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 106/212] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:36.025223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511362be-b287-40b0-88b8-721a2d862b34 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.065112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.135576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511362be-b287-40b0-88b8-721a2d862b34 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.172281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.173270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.173752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.174176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.190618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:36.198818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:36.200288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:36.201293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 107/213] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:36.206346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.212562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.275815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.313895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.321256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1380d8-7ae1-49c4-ad61-a87e6a3c9162 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.342096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c351e7-b240-4db6-ac77-37879d7172f5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.357877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.358516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.358787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.359051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.360281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:36.361885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:36.368277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 108/214] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:36.369165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.385203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.386505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.387365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.388070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.388933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:36.389658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:36.391615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:36.408351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 107/215] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:36.410366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.484353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d205-b45f-4752-8f2d-1a7743ebcb6a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.523145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.524191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.524656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.525680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.526658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:36.529052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:36.547633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b3226-28ab-415a-b1d8-caefd7aef7fc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.558415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 109/216] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:36 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:36.563513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.591838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.592500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.592776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.593047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] PATH_INFO: `/v3/users/11aecd0cdc054a15a3287fdee27b2191` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.609367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] RBAC: Authorizing `identity:get_user(user_id=11aecd0cdc054a15a3287fdee27b2191)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:36.610935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48562b-c944-4ee2-8e82-11b7f7c4725b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:36.612869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 108/217] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:36 2022] GET /identity/v3/users/11aecd0cdc054a15a3287fdee27b2191 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:36.618620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:36.645706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.744252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338c199-067b-42e5-8ce1-32ab75217634 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:36.785412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:36.786158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.786429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.786693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.804624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0742-606b-4368-a57b-6de2381188aa None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.805159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0742-606b-4368-a57b-6de2381188aa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.805159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf0742-606b-4368-a57b-6de2381188aa None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.807807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 109/218] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:36.816494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b10b8-b927-4c77-a7d7-ddb4640e4d0f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.816988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b10b8-b927-4c77-a7d7-ddb4640e4d0f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.817458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b10b8-b927-4c77-a7d7-ddb4640e4d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.842523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eb3b10b8-b927-4c77-a7d7-ddb4640e4d0f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:36.843885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eb3b10b8-b927-4c77-a7d7-ddb4640e4d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:36.885632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:36.888493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:36.937048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:36.962268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 110/219] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:36.971576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8e17-bc9e-4ca4-93d9-9b6608f1eb99 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.972213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8e17-bc9e-4ca4-93d9-9b6608f1eb99 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.972697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd8e17-bc9e-4ca4-93d9-9b6608f1eb99 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:36.974478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 111/220] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:36.982272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:36.982549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:36.982982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.008281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:37.009491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:37.059837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 110/221] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:37.065834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.097390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 112/222] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:37.172966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f9c5-5dd5-4ef8-8350-aa3906a12c5b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:37.194446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:37.209087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:37.209765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:37.210487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:37.210751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.211930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:37.213656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:37.215825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 111/223] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:37.221392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.286880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:37.384253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.396519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb768c58-b544-429e-92f5-210d1a8bd3c4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:37.461820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:37.462336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:37.462606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:37.462876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.463691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=11aecd0cdc054a15a3287fdee27b2191) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:37.463999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:37.465470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:37.479530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 113/224] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/role_assignments?user.id=11aecd0cdc054a15a3287fdee27b2191&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:37.480405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.534240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54995ac-f75c-461b-8322-774f5f7498a2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:37.613930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:37.614635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:37.614914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:37.615178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.643230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:37.652821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:37.654407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:37.655892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 112/225] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:37.664076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.729985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:37.732532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:37.749465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc569-a93e-42ef-87bd-7b4d272f8622 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:37.792557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70527182-3d67-4c67-9cdb-cd4e988bfe69 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:37.840674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:37.841299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:37.841598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:37.841866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.842553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:37.842848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:37.844277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:37.847654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:37.848311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:37.848607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:37.848854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:37.866677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 113/226] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:37.875834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:37.884503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:37.885488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:37.886343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:37.886966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 114/227] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:37.891007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:37.905436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.041375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17e744-f87f-47f0-908b-eeb0fc96e719 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.054093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:38.086321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.087063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.087406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.087740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.089060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.090640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.092954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 114/228] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:38.098911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.129386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d052-691c-42f8-b653-68ed48e8224b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.227361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.228394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.228901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.229183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.229910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:38.230213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.232631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.260690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 115/229] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:38.267636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c4995-b299-4386-a959-6c5298aeddce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.271424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.302958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:38.336040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.336691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.336967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.337241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.341511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.343036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.344744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 115/230] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:38 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:38.355067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.450604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:38.503891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a9d1c-4e99-4bad-b46c-330fe120d6f3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.532247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c71ae-e806-47c3-8c5a-0b02d1c18b5e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.585382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.585382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.585382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.585382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.585940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.585940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.585940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 116/231] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:38 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:38.588853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.589554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.589837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.590100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/b99ce6a157914c46a719b50446253f89/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.592963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.596183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, user_id=b99ce6a157914c46a719b50446253f89, role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.597708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.625888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 116/232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:38 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/b99ce6a157914c46a719b50446253f89/roles/1f37e1386b624e268925de2a48d79cc1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:38.712803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:38.740828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83ab7f-90e7-4534-8212-643767027732 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.790243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:38.790905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:38.791221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:38.791493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:38.792723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:38.794663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246feef-d414-4ed9-962d-809193fde5f6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:38.796731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 117/233] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:38 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:38.806359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:38.967414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383cc1ed-b311-4927-969a-4643b4343a14 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:38.981270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:39.043356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:39.044815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.045296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.045808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/b99ce6a157914c46a719b50446253f89/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:39.051616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, user_id=b99ce6a157914c46a719b50446253f89, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:39.054349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87e6ee-a72b-47ab-be1a-ba7a32b98cd7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:39.085911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 117/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:38 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/b99ce6a157914c46a719b50446253f89/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:39.312675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f317-ed5c-4143-afab-5a73ffcfa362 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.313645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f317-ed5c-4143-afab-5a73ffcfa362 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.314145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f317-ed5c-4143-afab-5a73ffcfa362 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:39.316075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 118/235] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:39.323142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.323587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.324018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:39.331909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42687c29-092f-4d91-8797-bf3ec0343ec6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.332208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42687c29-092f-4d91-8797-bf3ec0343ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.332471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42687c29-092f-4d91-8797-bf3ec0343ec6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:39.334052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 119/236] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:39.352147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.352493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.352698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:39.372576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:39.373834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:39.387256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:39.389296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:39.455736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 118/237] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:39.515254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 120/238] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:39.748061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:39.795958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:39.881927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:39.891919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db89aa-4e43-45e0-833f-2efc0438930e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:39.909922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:39.950169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2296d-e588-4342-81d5-a08643abaae9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:39.972476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:39.973523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:39.974235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:39.974504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.000079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:40.004934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:40.005856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:40.006663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:40.008696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 119/239] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:40.016726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:40.017756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.018224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.018686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.020018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:40.021699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:40.022573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:40.024177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 121/240] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:40.030534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:40.142452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c3ac2-2366-42dd-88b4-79f6735e16ac None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:40.185111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:40.186208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.186618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.187023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.188069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:40.189784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:40.225096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 122/241] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:40 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:40.237167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab30818-69b3-48f1-888a-fccbabb82b5c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:40.278027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:40.278917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.279338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.279765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.280592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:40.281024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:40.282736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe8674-1b2c-4a98-89c4-d897a4499b9b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:40.292975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 120/242] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:40.298995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:40.305949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:40.381327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab126-cd3f-4d6a-bd97-bf9a7328525b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.382090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab126-cd3f-4d6a-bd97-bf9a7328525b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.382562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab126-cd3f-4d6a-bd97-bf9a7328525b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.384507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 121/243] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:40.390388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.390988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.391456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.411041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:40.412558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:40.430228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:40.441370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cca58-8c77-4e3e-9b4e-3d0e34f666a5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:40.481735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:40.482754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.483271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.483842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] PATH_INFO: `/v3/users/1624f6a52eca4207800f8fc20bbbbcee` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.492372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 122/244] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:40.499211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] RBAC: Authorizing `identity:get_user(user_id=1624f6a52eca4207800f8fc20bbbbcee)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:40.501003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:40.503350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 123/245] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:40 2022] GET /identity/v3/users/1624f6a52eca4207800f8fc20bbbbcee => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:40.508992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:40.706194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850dede-85cb-4cca-99f1-17f217bc3ea7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:40.766849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:40.767820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:40.768273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:40.768726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:40.807754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:40.894232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:40.897826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.020290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 123/246] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:41.026009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.028573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.094031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.130485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.153629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959fa8b-6de1-483b-ba3c-86056dafe33a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.190779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2843da-c6e4-4e71-9a33-e97c0e8a2465 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.201205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.202393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.202944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.203498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.217338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:41.222969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:41.223862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:41.224790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:41.227190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 124/247] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:41.233224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.234404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.235638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.236457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.237967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.239166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.240654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.242715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 124/248] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:41.248816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.347294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992174fb-53d5-4946-bedf-82efaf177f45 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.358126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.386531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.387577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.388046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.388505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.389429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:41.389941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.391554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12525b37-f0b0-45e5-963f-ff1f0688c48c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.400953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 125/249] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:41.402649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785326d-2199-4583-8fb8-43c616a30fcd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.403221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785326d-2199-4583-8fb8-43c616a30fcd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.403684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785326d-2199-4583-8fb8-43c616a30fcd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.405403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 126/250] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:41.409002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785326d-2199-4583-8fb8-43c616a30fcd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.505315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fefaa-42df-4dca-883f-a9d2b5e2d244 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.520890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785326d-2199-4583-8fb8-43c616a30fcd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.570912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.572213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.573417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.574109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] PATH_INFO: `/v3/users/11aecd0cdc054a15a3287fdee27b2191` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.576346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] RBAC: Authorizing `identity:get_user(user_id=11aecd0cdc054a15a3287fdee27b2191)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.578904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80fda1-e25b-493b-b2db-5eb738185598 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.579676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.584262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 127/251] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/users/11aecd0cdc054a15a3287fdee27b2191 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:41.586287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469bc19-79c1-4277-b6e1-028fae31c510 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.586765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469bc19-79c1-4277-b6e1-028fae31c510 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.587224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469bc19-79c1-4277-b6e1-028fae31c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.589300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.589962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.590303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.590580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.591370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=1624f6a52eca4207800f8fc20bbbbcee) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:41.591686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.593232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.606912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 125/252] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/role_assignments?user.id=1624f6a52eca4207800f8fc20bbbbcee&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:41.607712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.608876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4469bc19-79c1-4277-b6e1-028fae31c510 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:41.610448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4469bc19-79c1-4277-b6e1-028fae31c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:41.699121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.744556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90f5c7-bd90-4f30-90ad-e10c9ae6bb49 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.782487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.783151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.783425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.783696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.784892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.786414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc37a-478d-4ba7-a239-2045ead6a380 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.788325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 126/253] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:41 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:41.794100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab791-db02-475c-8224-9806de77dd86 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.794491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab791-db02-475c-8224-9806de77dd86 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.794731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab791-db02-475c-8224-9806de77dd86 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.796505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 127/254] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:41.798105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab791-db02-475c-8224-9806de77dd86 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:41.799837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 128/255] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:41.803652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.803937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.804208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.842612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:41.843824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:41.871432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:41.916245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab791-db02-475c-8224-9806de77dd86 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:41.938519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 129/256] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:41.955219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:41.955899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:41.956211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:41.956483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/11aecd0cdc054a15a3287fdee27b2191/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:41.958452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, user_id=11aecd0cdc054a15a3287fdee27b2191, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:41.960231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:41.982319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 128/257] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:41 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/11aecd0cdc054a15a3287fdee27b2191/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:49:42.321940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.347574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.485740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bcc38-3979-4aba-8f70-8b76f789e603 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:42.511124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:42.522224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:42.527812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82045fec-8971-4f98-a7dd-4cf40a6a056d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:42.538702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:42.539042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:42.539256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:42.539518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:42.566609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:42.572218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:42.573105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:42.573942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 129/258] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:42.578985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.609991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:42.610849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:42.611141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:42.611521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:42.637204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:42.646654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:42.647584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:42.648398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:42.657701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 130/259] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:42.659573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.745979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca0fc5-3a20-4690-b904-bbc5f3a79760 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:42.810294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:42.810992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:42.811274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:42.811600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:42.812353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:42.812656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:42.814083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:42.823500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 130/260] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:42.829107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.874400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8832bc-2e0f-4f41-b58c-167ef6efffab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:42.902636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:42.914119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:42.914800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:42.915078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:42.915383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:42.916117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:42.916422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:42.918072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:42.927270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 131/261] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:42.935995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:42.966410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:43.025415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2a332-ca10-4225-b5b4-d8174464bf8e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:43.115594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:43.116243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.116544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.116790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.122250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:43.123728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:43.125691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 131/262] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:43.131236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:43.140537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3cde-d006-460b-9e8b-baa855128733 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:43.219294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:43.219955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.220227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.220492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] PATH_INFO: `/v3/users/b99ce6a157914c46a719b50446253f89` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.221699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=b99ce6a157914c46a719b50446253f89)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:43.223145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:43.224750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 132/263] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:42 2022] GET /identity/v3/users/b99ce6a157914c46a719b50446253f89 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:43.230577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:43.361480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d0ae20-a826-46db-b176-d2da3fdc9f8e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:43.383319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:43.424484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:43.437764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:43.438778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.440357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.441523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.446388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c0a76-1c8e-4f1c-a5aa-5a2a7adf5cf4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:43.523599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:43.524659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.525141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.525701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.554411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:43.559867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44229d0d-b394-4e9c-9027-cb3efd9cf2fa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:43.687499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:43.688920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b16821-5f69-4c20-91a5-7c75b076c07e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:43.771633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 132/264] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:43.772996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348901-ec58-43a6-b1ef-45ea6e05e933 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.773279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348901-ec58-43a6-b1ef-45ea6e05e933 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.773564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d348901-ec58-43a6-b1ef-45ea6e05e933 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.774983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 133/265] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:43.778617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d348901-ec58-43a6-b1ef-45ea6e05e933 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:43.809068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:43.873923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:43.897803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 133/266] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:43.899109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:43.899384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:43.899649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:43.929499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:43.931527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:43.993866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d348901-ec58-43a6-b1ef-45ea6e05e933 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.034162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.035252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.035628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.035990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.037356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:44.038310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 134/267] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:44.039149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb468a0-1e80-408e-b406-e1ef4dd9861b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:44.039816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.041382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 134/268] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:43 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:44.046657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d413b-4798-466e-9167-863fb20cbeed None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.046657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d413b-4798-466e-9167-863fb20cbeed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.046657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d413b-4798-466e-9167-863fb20cbeed None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.046657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 135/269] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:44.050537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d413b-4798-466e-9167-863fb20cbeed None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.157518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db71ca-283a-48a1-881a-c2692eea4e86 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.182024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:44.197939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.198926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.199300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.199666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.201096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:44.203103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee91da-f1a8-4a32-9980-39ba7bf2df0b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:44.205733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 135/270] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:44.207112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.207484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.207851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.229032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:44.230989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:44.279048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:44.319309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d413b-4798-466e-9167-863fb20cbeed None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.339420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 136/271] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:44.340353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.391948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.392563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.392842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.393111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.393870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=b99ce6a157914c46a719b50446253f89) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:44.394174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:44.395620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c217b-1eb2-400c-a9e7-0da86ac06cdc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:44.411886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 136/272] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/role_assignments?user.id=b99ce6a157914c46a719b50446253f89&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:44.423207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64915b-0195-4f44-a85b-dca286754280 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.423592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64915b-0195-4f44-a85b-dca286754280 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.423742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64915b-0195-4f44-a85b-dca286754280 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.425190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 137/273] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:44.439124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.439454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.439679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.471196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:44.472438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:44.563144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0d441-7a34-45f7-a906-b66c7996a4ce None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.588108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:44.612692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.613310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.614522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.614802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.615541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=b99ce6a157914c46a719b50446253f89) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:44.615902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:44.618507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:44.626634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 137/274] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/role_assignments?user.id=b99ce6a157914c46a719b50446253f89&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:44.627470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.635984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 138/275] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:44.665589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.796947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:44.810064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd843e3-dd8d-440d-b9f4-cefa6912b9c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.838440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0e142-2156-45d9-a47c-7fad6196fb7d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:44.853467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.854099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.854447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.854783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.872180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c1b7439c-8375-456f-9c4c-0c0990c46c99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:44.877595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:44.878515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:44.879319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 138/276] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:44.881066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de474283-daf1-4e2f-bbe0-3fcbab5bdda8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.881413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de474283-daf1-4e2f-bbe0-3fcbab5bdda8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.881775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de474283-daf1-4e2f-bbe0-3fcbab5bdda8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.883366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 139/277] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:44.888073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de474283-daf1-4e2f-bbe0-3fcbab5bdda8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:44.888289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:44.889280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.889556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.889915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.891146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:44.893853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb59f87-55a9-4b6c-90a9-49f562772b5f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:44.895833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 139/278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:44.897095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:44.897397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:44.898222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:44.922114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:44.923674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:44.949003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:44.995907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 140/279] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:44.996748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.020075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de474283-daf1-4e2f-bbe0-3fcbab5bdda8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.063751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.064407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.064676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.064934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.065611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:45.065908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.067979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.068445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.081502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 140/280] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.082328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.101400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af7fbc-9d09-4f54-b54d-7d88f4edccb8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.138134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.138996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.139444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.139913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.140812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.142550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.168321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 141/281] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:49:44 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:45.169915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.194670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cae427-c013-4d3e-bc0b-4b5629f220e1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.197101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.246353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.247324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.247739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.248182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.260904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:45.266317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:45.267316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:45.268101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:45.269700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 141/282] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:45.270699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.304201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317175be-3cd4-4a96-9621-2bd71ae2b264 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.336437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.350082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.351080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.351080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.352918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] PATH_INFO: `/v3/users/b3001a6fc3ce49c9a1df783d64276ff2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.370258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] RBAC: Authorizing `identity:get_user(user_id=b3001a6fc3ce49c9a1df783d64276ff2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.371755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.373493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 142/283] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/users/b3001a6fc3ce49c9a1df783d64276ff2 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.374469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.398155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2176b1-1e64-4e4e-996b-8b597382b6ce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.448317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.448972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.449276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.449524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.461220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:45.467556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:45.468661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:45.469480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 142/284] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:45.473778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.502945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.535235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5e7f8-7c42-45f9-8e17-63465fb17e59 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.602737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.604579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.605294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.606020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.608452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:45.609181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.613371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.620475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b4afc-1ea9-4f8c-8365-a4b5b43c3706 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.632621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 143/285] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.642198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.674186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.675500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.676195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.676608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.677789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, user.id=b3001a6fc3ce49c9a1df783d64276ff2, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:45.678248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.680771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.689209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 143/286] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/role_assignments?user.id=b3001a6fc3ce49c9a1df783d64276ff2&scope.system=all&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.690515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.711055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.819859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306298e-a2d8-4cbc-839a-aca145f831bd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.870054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:45.892488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a73c0-8a76-4033-b09d-d3662eaf45f3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:45.911823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.912898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.913319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.914148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.915268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:45.915738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.917901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.930429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 144/287] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.935686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:45.938475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:45.939239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:45.939582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:45.939917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] PATH_INFO: `/v3/users/1624f6a52eca4207800f8fc20bbbbcee` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:45.941146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] RBAC: Authorizing `identity:get_user(user_id=1624f6a52eca4207800f8fc20bbbbcee)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:45.942749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:45.944607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 144/288] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/users/1624f6a52eca4207800f8fc20bbbbcee => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:45.950530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:46.058290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c40-119f-4d8b-9cd6-fc3ee38433f7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.073181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:46.105896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.106866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.107327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.108297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] PATH_INFO: `/v3/users/11aecd0cdc054a15a3287fdee27b2191` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.109689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] RBAC: Authorizing `identity:get_user(user_id=11aecd0cdc054a15a3287fdee27b2191)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.111368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:46.113507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 145/289] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/users/11aecd0cdc054a15a3287fdee27b2191 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:46.122740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:46.163466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e70f4-29ac-4e48-a19c-815e18472964 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.232082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.234558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.234835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.235104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.237525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.239980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:46.241880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 145/290] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:45 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:46.247479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:46.286710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c1a96-22ec-4211-9f9e-4997a7c222d5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.311271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:46.328050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.328676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.328949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.329217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.397403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:46.424833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.429573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:46.527724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0f270-ea38-4a73-a059-cd6328e76130 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.551255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 146/291] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:46.557564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:46.618270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.618952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.619291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.619559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/1624f6a52eca4207800f8fc20bbbbcee/roles/bdb8bd48f0394d9bb06f750023d9239f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.625460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, user_id=1624f6a52eca4207800f8fc20bbbbcee, role_id=bdb8bd48f0394d9bb06f750023d9239f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.631366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:46.659587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 146/292] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:49:46 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/users/1624f6a52eca4207800f8fc20bbbbcee/roles/bdb8bd48f0394d9bb06f750023d9239f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:46.718739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:46.749338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d5cef-ef4f-4a8a-a5f4-e1c73b95c683 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.806438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.807160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.807437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.807707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.809029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.810534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79cd80-e36b-4bdb-b776-253bbc375967 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:46.812740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 147/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:46 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:46.818884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:46.932628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:46.954108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c76b7-3fe4-4c7f-ba04-8777e36f85ba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:46.992688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:46.993765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:46.994050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:46.994378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:46.995160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=11aecd0cdc054a15a3287fdee27b2191) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:46.995463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:46.996965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289f5e-f66e-433a-b09c-c329ff6e08d8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:47.004148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 147/294] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:46 2022] GET /identity/v3/role_assignments?user.id=11aecd0cdc054a15a3287fdee27b2191&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:47.510906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1d82-f052-46a5-9926-8a90eba0796b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.511427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1d82-f052-46a5-9926-8a90eba0796b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.511837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b1d82-f052-46a5-9926-8a90eba0796b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.513617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 148/295] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:47.519356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.520258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.520721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.545950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:47.547487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:47.572562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bf12-49c4-40ad-9f12-fbff68cb4c74 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.573068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bf12-49c4-40ad-9f12-fbff68cb4c74 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.573563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2bf12-49c4-40ad-9f12-fbff68cb4c74 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.579162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 149/296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:47.582104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bab3c-38a5-4f8d-ada0-feeae6a8fd3e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.582636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bab3c-38a5-4f8d-ada0-feeae6a8fd3e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.583113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bab3c-38a5-4f8d-ada0-feeae6a8fd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.609664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-692bab3c-38a5-4f8d-ada0-feeae6a8fd3e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:47.611858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-692bab3c-38a5-4f8d-ada0-feeae6a8fd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:47.631270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 148/297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:47.690225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 150/298] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:47.907442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:47.959434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69479-f0a3-4c26-90cb-637b7d9b31b3 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.960400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69479-f0a3-4c26-90cb-637b7d9b31b3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.960863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f69479-f0a3-4c26-90cb-637b7d9b31b3 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.962946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 151/299] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:47.968460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:47.969025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:47.969570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:47.991828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:47.993303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:47.998636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.026978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80c572-e6af-4263-aa44-1f73736a7e87 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.063447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.064067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.064341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.064616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.069836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 152/300] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:48.071051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.080887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.086588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:48.094665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:48.095735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:48.103734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 149/301] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:48.104805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.191150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd7892-8cf5-4a06-8905-476d56bab331 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.228412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.229607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.230086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.230551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.231877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.233518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.235812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 153/302] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:48.243897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33cc8b2-6448-4c05-85f9-670c3ba0b2de None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.244996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.287355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.287975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.288246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.288513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.289190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:48.289744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.292119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.295149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.307318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 150/303] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:48.310885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.342609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.373465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aefb6-5f21-4f34-b065-95090e6ebdb6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.423543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.424219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.424507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.425027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] PATH_INFO: `/v3/groups` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.426298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.427852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.442966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dbe14-34dd-4ac2-a7a8-da26c6ba1a66 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.449881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 154/304] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 17 19:49:48 2022] POST /identity/v3/groups => generated 229 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:49:48.458763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.486325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.487471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.487955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.488429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] PATH_INFO: `/v3/users/c9a6eb8400ed437ab00967f88ef71e0b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.505416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a6eb8400ed437ab00967f88ef71e0b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.507508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.510089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 151/305] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/users/c9a6eb8400ed437ab00967f88ef71e0b => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:48.517606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.587262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ceb3d0-c990-4b2c-aa4c-b84104f70aad None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.615997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.631236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.632176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.632461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.632797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.634856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:48.647204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:48.652391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:48.653632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:48.654604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:48.660271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 155/306] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:48.661119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.678614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af713f0-a79d-48b5-9932-b4fa5b27dc24 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.720729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.721977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.722255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.722524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.723261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, user.id=c9a6eb8400ed437ab00967f88ef71e0b, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:48.723564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.729559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.734339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 152/307] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/role_assignments?user.id=c9a6eb8400ed437ab00967f88ef71e0b&scope.system=all&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:48.787411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1d05f-e9be-47f3-b56c-2ea451e6d0a3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:48.839222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:48.840017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:48.840441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:48.840930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:48.842019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:48.842477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:48.844710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:48.857186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 156/308] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:48.864603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:48.979781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389238-5646-4a2e-ace5-e3a1e0bde9ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:49.001428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:49.023837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:49.024501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.024779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.025044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] PATH_INFO: `/v3/users/b3001a6fc3ce49c9a1df783d64276ff2` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.026332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] RBAC: Authorizing `identity:get_user(user_id=b3001a6fc3ce49c9a1df783d64276ff2)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:49.027815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97f206-a3d6-4b11-8347-834f12d9371a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:49.029792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 153/309] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:48 2022] GET /identity/v3/users/b3001a6fc3ce49c9a1df783d64276ff2 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:49.035335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:49.057184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f55c-5e1c-4380-8fad-0272227b5323 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.057626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f55c-5e1c-4380-8fad-0272227b5323 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.057874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35f55c-5e1c-4380-8fad-0272227b5323 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.059085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 154/310] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:49.064694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.064992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.065215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.086714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:49.087990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:49.096811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:49.143783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dff01-b646-4599-adff-a5f9992f0279 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:49.173009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 155/311] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:49.188824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:49.190013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.190383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.190638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] PATH_INFO: `/v3/system/users/b3001a6fc3ce49c9a1df783d64276ff2/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.192257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3001a6fc3ce49c9a1df783d64276ff2, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:49.193745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:49.205892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 157/312] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 17 19:49:49 2022] PUT /identity/v3/system/users/b3001a6fc3ce49c9a1df783d64276ff2/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:49.449060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:49.577562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f0505-dd2b-475f-a0ee-5c6b17e7f13a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:49.591133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:49.611346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:49.612001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.612282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.612550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.624404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:49.629198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:49.630140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:49.631164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 156/313] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:49.638508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:49.752913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ebb7-e04c-4f47-994d-d0da30cf8daa None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:49.774308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:49.789614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:49.790540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.791063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.791702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.792650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:49.793144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:49.795318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:49.805386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 158/314] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:49.813164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:49.919995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec3ff6-d42c-41a5-997a-c76dcb38f6f2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:49.958727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:49.959675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:49.960127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:49.960568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] PATH_INFO: `/v3/users/1624f6a52eca4207800f8fc20bbbbcee` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:49.961927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] RBAC: Authorizing `identity:get_user(user_id=1624f6a52eca4207800f8fc20bbbbcee)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:49.963651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:49.965439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 157/315] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:49 2022] GET /identity/v3/users/1624f6a52eca4207800f8fc20bbbbcee => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:49.973909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:50.080546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b59d3-4ff0-484e-9780-35587c651932 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:50.120083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:50.121040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.121495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.121969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.185932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:50.198268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:50.200289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:50.301737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 159/316] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:50.310779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:50.364751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:50.420776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aff435-70e2-45cc-8ed4-509a74372b47 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:50.455496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:50.456428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.456862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.457262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.458607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:50.460202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c209e-6ec8-444c-a309-86daeb0420c8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:50.462482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 158/317] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:50 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:50.471834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:50.580474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:50.599374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a106736-aa82-4950-9a5c-b5dc918f53b5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:50.634905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:50.635441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.635716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.635986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.636728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, scope.project.id=80efcdefebdf4680bcd88df222732e4f, user.id=1624f6a52eca4207800f8fc20bbbbcee) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:50.637144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:50.638710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9f397-6f0b-4120-aa69-d7e498e927b6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:50.645660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 160/318] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:49:50 2022] GET /identity/v3/role_assignments?user.id=1624f6a52eca4207800f8fc20bbbbcee&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:50.719049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ecc6-2267-48f8-9322-aca2e4a7d1bf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.720038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ecc6-2267-48f8-9322-aca2e4a7d1bf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.720520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ecc6-2267-48f8-9322-aca2e4a7d1bf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.722512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 159/319] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:50.728737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee176c-4f30-43e9-9151-76042ffb2025 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.729347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee176c-4f30-43e9-9151-76042ffb2025 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.729856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee176c-4f30-43e9-9151-76042ffb2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.749819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6bee176c-4f30-43e9-9151-76042ffb2025 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:50.751361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6bee176c-4f30-43e9-9151-76042ffb2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:50.833096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 161/320] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:50.841906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6de8-6308-4cfa-8e39-305254c57a90 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.842282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6de8-6308-4cfa-8e39-305254c57a90 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.842628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6de8-6308-4cfa-8e39-305254c57a90 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.844264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 160/321] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:50.849667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:50.850066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:50.850408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:50.874173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:50.875435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:50.950561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 162/322] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:51.105579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a6de8-6308-4cfa-8e39-305254c57a90 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.210365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a6de8-6308-4cfa-8e39-305254c57a90 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.217836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.225807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.227940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.250103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.250740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.251022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.251282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.252370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:51.253865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123e4d-c1ac-474b-90e5-c4b278e11892 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:51.255439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 161/323] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:51.255439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0bed1-2096-4e32-b034-7587dd0e4584 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.255439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0bed1-2096-4e32-b034-7587dd0e4584 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.255439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0bed1-2096-4e32-b034-7587dd0e4584 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.255439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 162/324] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:51.261032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0bed1-2096-4e32-b034-7587dd0e4584 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.321483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d16d2-a848-425c-95f7-246bf96aec94 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.356656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.357295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.357575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.357868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.359821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0bed1-2096-4e32-b034-7587dd0e4584 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.367858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-08dd664a-ba96-4809-bc57-708dc987f9fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:51.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:51.373744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:51.374543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 163/325] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:51.375317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.375560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.375800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.393044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:51.394280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:51.400253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.400913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.401185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.401444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] PATH_INFO: `/v3/groups` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.402089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:51.403550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:51.421565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 163/326] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 17 19:49:51 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:49:51.422402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.465017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.470988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.476752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 164/327] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:51.545643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d24e2-a0ca-4f7b-b304-0eefdbb6ce74 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.589381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.589712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.589821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.590057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.590735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:51.591033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:51.592448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:51.602380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 164/328] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:51.608334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.722486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eccf8ee-a78c-43a7-98af-5d3ee1973883 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.741084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.768438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.769088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.769362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.769662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] PATH_INFO: `/v3/users/c9a6eb8400ed437ab00967f88ef71e0b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.770929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a6eb8400ed437ab00967f88ef71e0b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:51.772494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:51.774704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 165/329] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/users/c9a6eb8400ed437ab00967f88ef71e0b => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:51.780513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.862590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a721fd7-2d26-4261-9592-25079528f21c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.865648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.871172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:51.921853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.923368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.923368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.923368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.930177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9f2e3-7416-4854-b535-8ce0c15b67a6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:51.937223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:51.942644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:51.943533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:51.944313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:51.944907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 165/330] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:49:51.950859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:51.983596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:51.984325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:51.984690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:51.985028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] PATH_INFO: `/v3/system/users/c9a6eb8400ed437ab00967f88ef71e0b/roles/bdb8bd48f0394d9bb06f750023d9239f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:51.988410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c9a6eb8400ed437ab00967f88ef71e0b, role_id=bdb8bd48f0394d9bb06f750023d9239f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:51.989057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:52.003461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 166/331] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 17 19:49:51 2022] PUT /identity/v3/system/users/c9a6eb8400ed437ab00967f88ef71e0b/roles/bdb8bd48f0394d9bb06f750023d9239f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:49:52.091176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f564b-59cb-4eb7-805c-c9c70b60e2e5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:52.131589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:52.132565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:52.132984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:52.133506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:52.134423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:52.134918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:52.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:52.148248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 166/332] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:52.154557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:52.268489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d11aa4-7664-4bb4-9bf9-92d963b595f5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:52.303009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:52.311452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:52.312756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:52.313280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:52.313572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] PATH_INFO: `/v3/users/b3001a6fc3ce49c9a1df783d64276ff2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:52.314753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=b3001a6fc3ce49c9a1df783d64276ff2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:52.316295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d6956-0b99-4951-8cfc-2113d65e6bb7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:52.321760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 167/333] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:52 2022] GET /identity/v3/users/b3001a6fc3ce49c9a1df783d64276ff2 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:52.324589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:52.359156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:52.451131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e451b-9b65-454f-90dd-d0ccdafe786e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:52.491719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:52.492336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:52.492615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:52.492885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:52.493649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] RBAC: Adding query filter params (role.id=8f4a64d3f31545619fe7a9e422d55398, user.id=b3001a6fc3ce49c9a1df783d64276ff2, scope.system=all) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:52.493956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:52.495622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f9db9-bc85-470c-9dd7-acff0e222d2f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:52.510655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 167/334] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 17 19:49:52 2022] GET /identity/v3/role_assignments?user.id=b3001a6fc3ce49c9a1df783d64276ff2&scope.system=all&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:53.891246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd764-4cf6-482e-9321-7da1fce7e4a0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:53.891812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd764-4cf6-482e-9321-7da1fce7e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:53.892115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd764-4cf6-482e-9321-7da1fce7e4a0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:53.895051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfbc6e-10e4-4104-abcd-eb4172f1a600 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:53.895447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfbc6e-10e4-4104-abcd-eb4172f1a600 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:53.895614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfbc6e-10e4-4104-abcd-eb4172f1a600 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:53.897227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 168/335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:53.897450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 168/336] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:53.902262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166a97-498c-4539-ab1d-e4c0c346b728 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:53.902546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166a97-498c-4539-ab1d-e4c0c346b728 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:53.902820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a166a97-498c-4539-ab1d-e4c0c346b728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:53.908808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d0f85-f207-46ba-806a-e8ebb9b65bf7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:53.909453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d0f85-f207-46ba-806a-e8ebb9b65bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:53.909759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d0f85-f207-46ba-806a-e8ebb9b65bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:53.926646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8a166a97-498c-4539-ab1d-e4c0c346b728 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:53.928993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8a166a97-498c-4539-ab1d-e4c0c346b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:53.939053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-649d0f85-f207-46ba-806a-e8ebb9b65bf7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:53.940823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-649d0f85-f207-46ba-806a-e8ebb9b65bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:54.067026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 169/337] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:54.075703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 169/338] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:49:54.479905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e1a1-d9cc-4182-b173-c689bb1720ca None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:54.480351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e1a1-d9cc-4182-b173-c689bb1720ca None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:54.480460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d4e1a1-d9cc-4182-b173-c689bb1720ca None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:54.482224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 170/339] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:54.489954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425736-c02c-442d-b03d-08800435204a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:54.490226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425736-c02c-442d-b03d-08800435204a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:54.490536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425736-c02c-442d-b03d-08800435204a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:54.523341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-18425736-c02c-442d-b03d-08800435204a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:54.524188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4e1a1-d9cc-4182-b173-c689bb1720ca None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:54.525290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-18425736-c02c-442d-b03d-08800435204a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:54.547163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:54.660061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 170/340] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 17 19:49:54.723199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d4e1a1-d9cc-4182-b173-c689bb1720ca None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:54.760242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:54.768231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18425736-c02c-442d-b03d-08800435204a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:54.773080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:54.773790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:54.774072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:54.774343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:54.803914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:54.808870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:54.809823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:54.810650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 171/341] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:54.816626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:54.983081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18425736-c02c-442d-b03d-08800435204a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.046448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05d67-4d14-45e7-805a-a8d7f87e28f1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.063771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.064899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.065362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.066042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.090660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:55.099015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:55.099959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:55.100755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:55.107466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 171/342] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:55.109940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.119938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.133462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.134264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.134569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.134854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.135663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:55.135950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:55.137377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:55.159894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 172/343] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:55.160727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.259548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.297742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf90953-12f3-40c6-8296-3500acbf371a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.334264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775d27a-39aa-480d-9b9b-b10c75aeab53 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.344225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.344879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.345153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.345425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.361243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:55.367174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:55.368141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:55.369021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 172/344] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:55.369755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.385597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.386609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.387245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.387519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.388319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:55.388735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:55.390593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541babb2-e78b-4317-8dec-e1eeff77b82c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:55.400132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 173/345] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:55.401915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0eda1-b141-4eff-914f-36aa1ffffdd0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.402263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0eda1-b141-4eff-914f-36aa1ffffdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.402603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0eda1-b141-4eff-914f-36aa1ffffdd0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.404135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 174/346] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:55.405019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0eda1-b141-4eff-914f-36aa1ffffdd0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.472281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.524968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395cd2a-4dea-40f0-b922-1ae929eb3068 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.531592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0eda1-b141-4eff-914f-36aa1ffffdd0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.548050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.576740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.577689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.578132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.578561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.590674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.591330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.591614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.591892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.593521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:55.593855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:55.595706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dd8f1-bd63-454c-bed9-c1848ba214b3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:55.610352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 175/347] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:55.615745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.615745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.615745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.638695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:55.640245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:55.674846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:55.676744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:55.741569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 176/348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:55.742999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.765922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.816088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 173/349] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:55.817511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:55.877980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d2ade-306c-493d-9fc0-920d2ffebc71 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:55.880253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:55.931126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:55.932081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:55.932421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:55.932934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:55.957668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82eed8-9a15-4105-bd26-8e4026ccebe5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.000665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.001476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.001823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.002093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] PATH_INFO: `/v3/users/c9a6eb8400ed437ab00967f88ef71e0b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.003376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a6eb8400ed437ab00967f88ef71e0b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.004958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3645b2-cd77-4fd7-be4a-9d2e6dfe2e81 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.006799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 174/350] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/users/c9a6eb8400ed437ab00967f88ef71e0b => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.007955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831af6-2933-4fec-971b-97b813427b63 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.008224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831af6-2933-4fec-971b-97b813427b63 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.008488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e831af6-2933-4fec-971b-97b813427b63 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.010044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 175/351] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:56.010737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831af6-2933-4fec-971b-97b813427b63 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.018897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.020379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.055162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.136412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 177/352] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:56.137268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.142833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.169776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e831af6-2933-4fec-971b-97b813427b63 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.217660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.218294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.218563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.218823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.219932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.221564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a72279-70c8-4049-a563-5a6561db79cd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.223219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 176/353] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.224355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.224570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.224833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.246222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:56.247417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:56.262269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7955cfc-2198-4d41-83b4-1e7bd336c28c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.313337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.314453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.315003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.315441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.316616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] RBAC: Adding query filter params (role.id=bdb8bd48f0394d9bb06f750023d9239f, user.id=c9a6eb8400ed437ab00967f88ef71e0b, scope.system=all) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:56.317555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.319684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.332633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 177/354] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:56.333665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.335291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 178/355] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/role_assignments?user.id=c9a6eb8400ed437ab00967f88ef71e0b&scope.system=all&role.id=bdb8bd48f0394d9bb06f750023d9239f => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.336482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.369963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.438079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.473141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c8534-2f09-4b3f-90ce-6dfaac18e628 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.477507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae14580-2cc9-4a17-83cc-2e2faa57873f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.532517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.533442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.533900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.534298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.540629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.542869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.543192 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.543469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.545441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.546871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.548901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 179/356] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.549737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.551831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:56.556882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:56.558002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:49:56.558984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:56.562188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 178/357] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:56.563255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.690055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.691704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba3301-c60c-42df-be7b-36fdc2c88a7a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.703653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c5971-70e3-433e-84dc-97f9b669742c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.727926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:56.741903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.743269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.743269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.743269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.749765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.751230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.752572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.753042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 180/358] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.753291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.757680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.757803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.757878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.758653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.760532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7632b-c909-42af-b93b-6b78bd6b6875 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.763049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 179/359] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.763816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af82ea-9339-437e-a6a7-45ea3c907c6f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.764101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af82ea-9339-437e-a6a7-45ea3c907c6f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.764371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af82ea-9339-437e-a6a7-45ea3c907c6f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.765860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 180/360] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:56.770338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af82ea-9339-437e-a6a7-45ea3c907c6f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.880748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a768a62-89a2-4598-a860-109bf2b18382 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.892896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af82ea-9339-437e-a6a7-45ea3c907c6f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:56.925977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.926429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.926711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.927434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.929665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:56.929909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.931295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.935832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:56.936520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.937279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.937636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.938665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=0507e9f171104361a7ff74d326317861) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:56.939091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:56.940706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec060329-dadc-4cb9-bef3-b3fa966335cc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:56.941258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 181/361] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.942491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:56.946514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 181/362] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:56.947858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:56.948212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:56.948479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:56.968625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:56.969904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:56.989911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.020064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.062577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 182/363] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:57.064271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.074442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41bc8-6a66-4f0b-b4b8-124a0d778ab1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.113567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.114273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.114554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.114824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.129112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:57.134108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:57.135122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 182/364] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:49:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:57.136503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.193883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692b3ee-f05c-429d-9070-8a968e32d089 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.241807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.242418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.242689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.242948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.243683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=0507e9f171104361a7ff74d326317861) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:57.244016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.245499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.251675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 183/365] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:57.252706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.271155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043144e-52c9-481e-b3e1-670691bde71f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.273952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.291053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.317912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.319167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.319652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.320126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.368653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7f74f-1a77-49cf-81ef-4689bf3b7011 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.403886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.406259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.406730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.407359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.407630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.407893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.408580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:57.408874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.411139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.421393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 184/366] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:57.422116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.516490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.544688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 183/367] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:57.545975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.547209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639cba0-e2e1-48d5-950d-3481e32d4f48 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.579492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.587532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.588266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.588606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.588937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.602364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:49:57.607854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:49:57.608831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:49:57.609689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:49:57.610776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 185/368] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:49:57.611940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.686558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf06c8-b0fb-424e-bdeb-d4484c72e8a3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.726146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.727246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.727720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.728194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.740668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cadd0-62fe-40ac-809d-3f604f0324c4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.784991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.785570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.785850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.786121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.789194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.789383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.792844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 186/369] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:57.794858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.812778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.830005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.832101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.860325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:57.934984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed05926-e871-423f-81d8-8a7d0f397355 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:57.974834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 184/370] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:57.976097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:57.977946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:57.978639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:57.978964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:57.979240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:57.980037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:57.980343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:57.981906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:57.991357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 187/371] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:57.992406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:58.071885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:58.086784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6e8d3-8ba0-49c1-a70c-abcd6933ae3b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.101897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:58.121726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ec0dc-2c31-4fc7-84e5-c2468405c864 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.125676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.126448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.126818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.127179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.128410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.129972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.131979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 185/372] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:58.132858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:58.167233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.168249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.168776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.169681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.171115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.172871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.177317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 188/373] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:57 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:58.177317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:58.279706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd8319-156f-420a-81d9-e204bf5205ec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.298498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec97c-d4e3-4c5e-b67a-e30bb1413538 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.343190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.343718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.344159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.344429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.345706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=80efcdefebdf4680bcd88df222732e4f) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:58.346236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.348728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.352022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:58.355410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 189/374] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:49:58 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:58.355551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.356214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:58.362842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.363327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.363788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.471488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b22a3f-cfcf-4e1a-bc8a-c0747aee8ba7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.510755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.511562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.511844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.512117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.513313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.515790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.518621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 190/375] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:49:58 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:58.524939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:58.529166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:58.597706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.600998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.610266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:58.646649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2e277-4440-47a5-91d5-9e3dfa17f1b6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:58.686604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:58.687366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:58.687648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:58.687916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:58.688663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=80efcdefebdf4680bcd88df222732e4f) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:58.688971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:58.690655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:58.698903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 191/376] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:49:58 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:49:58.884096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 186/377] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:49:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:49:58.891151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:59.032345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e765025-0dc1-48d1-bf4c-cde60ff9ef8e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:59.098778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:59.110130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:59.111930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.112373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.112837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] PATH_INFO: `/v3/projects/79e7870848b34ea38cb8567a241b0ce4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.117012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] RBAC: Authorizing `identity:get_project(project_id=79e7870848b34ea38cb8567a241b0ce4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:59.119859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:59.123484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 192/378] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:49:58 2022] GET /identity/v3/projects/79e7870848b34ea38cb8567a241b0ce4 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:59.129490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:59.298301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:59.340944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c2fda-8f22-4260-b46d-9f827d18674d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:59.416456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:59.418046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.418541 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.419370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] PATH_INFO: `/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.428351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] RBAC: Authorizing `identity:get_group(group_id=ca34b16edee14ec9be3aeafecc9ae48b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:59.431241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb4495-cd83-411f-9e64-662fc92439a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:59.434907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 187/379] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:49:59 2022] GET /identity/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b => generated 229 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:59.440571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:49:59.482686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15973b-25f3-4157-bc8c-2901a9a883ef None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.483000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15973b-25f3-4157-bc8c-2901a9a883ef None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.483232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15973b-25f3-4157-bc8c-2901a9a883ef None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.484857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 188/380] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:49:59.491275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ef21-3509-4c1f-aa50-d8ad6b530853 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.491557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ef21-3509-4c1f-aa50-d8ad6b530853 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.491828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4ef21-3509-4c1f-aa50-d8ad6b530853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.513028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-87c4ef21-3509-4c1f-aa50-d8ad6b530853 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:59.514717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-87c4ef21-3509-4c1f-aa50-d8ad6b530853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:59.561116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72ae29-7c00-4330-b716-9d2b5ad645f9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:49:59.601217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:49:59.601858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.602134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.602397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.603279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] RBAC: Adding query filter params (group.id=ca34b16edee14ec9be3aeafecc9ae48b, role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=79e7870848b34ea38cb8567a241b0ce4) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:49:59.603584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:49:59.605159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acd380-df02-4f39-9631-8364d1ce1abf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:49:59.615213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 189/381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:59.616760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a40a5-5e3f-4436-9db2-1f8e79e5aa64 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.616873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 193/382] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:49:59 2022] GET /identity/v3/role_assignments?group.id=ca34b16edee14ec9be3aeafecc9ae48b&scope.project.id=79e7870848b34ea38cb8567a241b0ce4&role.id=c974a286ee1942c8bab82b232c6af142 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:49:59.617265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a40a5-5e3f-4436-9db2-1f8e79e5aa64 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.622004 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a40a5-5e3f-4436-9db2-1f8e79e5aa64 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.624052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 190/383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:49:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:49:59.639638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:49:59.639924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:49:59.640200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:49:59.670723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:49:59.671961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:49:59.754138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:49:59.762969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 194/384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:49:59.938771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a40a5-5e3f-4436-9db2-1f8e79e5aa64 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.031149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:00.060747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a40a5-5e3f-4436-9db2-1f8e79e5aa64 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.116595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.117646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.118133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.118588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.133226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:00.138458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:00.139386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:00.140269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:00.143245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 191/385] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:49:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:00.150287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.166713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.288819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cbdcf-c216-4611-a11f-3aacbced1367 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.290123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b2b85-fc27-405d-93c5-28cf553a6995 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.333854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.334488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.334760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.335024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.335716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:00.336011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.337406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.347296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 195/386] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:00.352991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.363625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.364950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.365488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.365992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.370883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:00.388601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:00.394207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:00.395096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:00.395883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:00.398618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 192/387] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:00.405933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.417492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:00.462536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607cfa1-4288-443c-b841-08d75f9e8f3b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.496914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.497575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.497848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.498113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.499264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.500840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.502783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 196/388] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:00.509466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.584571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2357c53-411d-4f97-9e46-0474568bebbb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.630256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f7d56-cd48-432b-9012-3a3cc8c5a86f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.633562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.634214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.634529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.634805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.635486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:00.635791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.637182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.655474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:00.663671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 193/389] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:00.666631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.667519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.667912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.668324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.669953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.670154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.671797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.673677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 197/390] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:00.679429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:00.784487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9832c-6365-41e5-add2-984e72f2dacb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.798018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:00.819743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.820521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.820743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.821010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/groups/636f259ffddb4a96a0c83eb9393929f5/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.823097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, group_id=636f259ffddb4a96a0c83eb9393929f5, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.824555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.843569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 198/391] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 17 19:50:00 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/groups/636f259ffddb4a96a0c83eb9393929f5/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:00.871397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6f79-c2ad-42df-9bcd-d9647c8ff2ad None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:00.943959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:00.944645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:00.944919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:00.945193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:00.946483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:00.948005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:00.950037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 194/392] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:00.955685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:01.024501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:01.083486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1e04f-a156-4c9b-afe7-91597de93e25 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:01.121443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:01.122181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.122456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.122723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.123842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:01.125876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e365ce-6f56-4b2d-b00a-ebe60c95d35b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:01.127897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 199/393] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:00 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:01.134075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:01.218707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0681-435d-4ab1-9ad6-ec6275031de7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.219263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0681-435d-4ab1-9ad6-ec6275031de7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.219475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a0681-435d-4ab1-9ad6-ec6275031de7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.222274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 200/394] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:01.227245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a294957-eb2b-4f18-ab34-412d4bf4a7e2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.227532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a294957-eb2b-4f18-ab34-412d4bf4a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.227801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a294957-eb2b-4f18-ab34-412d4bf4a7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.249712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a294957-eb2b-4f18-ab34-412d4bf4a7e2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:01.250900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a294957-eb2b-4f18-ab34-412d4bf4a7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:01.256793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c58d2-3772-4ee8-8a00-130302d84e09 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:01.329976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 201/395] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:01.333954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:01.356764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:01.357809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.358389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.358934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861/groups/636f259ffddb4a96a0c83eb9393929f5/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.366309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debaaae-eb9d-4123-b2ae-36f83b10fcb0 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.366587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debaaae-eb9d-4123-b2ae-36f83b10fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.366812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] RBAC: Authorizing `identity:create_grant(project_id=0507e9f171104361a7ff74d326317861, group_id=636f259ffddb4a96a0c83eb9393929f5, role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:01.367237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debaaae-eb9d-4123-b2ae-36f83b10fcb0 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.368421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 202/396] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:01.369196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:01.378836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.379240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.379502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.389404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 195/397] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 17 19:50:01 2022] PUT /identity/v3/projects/0507e9f171104361a7ff74d326317861/groups/636f259ffddb4a96a0c83eb9393929f5/roles/1f37e1386b624e268925de2a48d79cc1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:01.406224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:01.407884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:01.473637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:01.504431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 203/398] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:01.642823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:01.785737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389826e-dcb3-4e0b-93d2-b564c065a79f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:01.830660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:01.855732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:01.856381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:01.856657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:01.856923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:01.876544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:01.883023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:01.883902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:01.884759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 196/399] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:01.891603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:01.978034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.029121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281c507-6de0-4ac2-9855-03f61d089d56 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.033082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03604fd6-e939-474b-a70e-2f6fa08fc574 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.081576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.082219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.082488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.082754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.089148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.089814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.090086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.090356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.091027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:02.091325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.092698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.098897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:02.103857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:02.105043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:02.106146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:02.106757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 197/400] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:50:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:02.106837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 204/401] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:02.111117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.115880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.181225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.250060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0f54-8587-4891-ab8a-18da087769ee None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.271464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.289922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ce108-767a-4db7-a2db-a85d49260dc4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.327093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.327755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.328034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.328306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.331061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.331842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.332121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.332380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.333103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:02.333455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.334304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.334951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.335768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.338003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 198/402] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:02 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:02.346439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.349081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 205/403] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:02.355217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.476483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ab0d-1537-4f86-a405-eed9bdf1b30b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.478134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.517370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.517966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.518320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.519602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.520769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.522270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.524180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 206/404] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:02 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:02.530422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.577599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2927037-1917-44b9-8a88-9868601c5a05 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.615266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.616358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.616835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.617318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.618832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.620685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838183a1-8adc-4e39-9d9e-f11ace104fff None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.623223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 199/405] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:02 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:02.624942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12692f-2db5-4917-b689-575b39c9eff8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.625428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12692f-2db5-4917-b689-575b39c9eff8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.625912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12692f-2db5-4917-b689-575b39c9eff8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.631546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 200/406] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:02.632559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12692f-2db5-4917-b689-575b39c9eff8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.648279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9931b6-836f-4626-8ff9-9ab3d656cbb0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.685607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.687948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.688284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.688698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.689007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.690448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.691991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3a1c0-6c2c-40ba-9ce3-900bf1502aec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.699526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 207/407] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:02 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:02.701055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.701554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.701744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.722225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:02.723268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:02.750130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12692f-2db5-4917-b689-575b39c9eff8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.756656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:02.793270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.794776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.795359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.795839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/groups/636f259ffddb4a96a0c83eb9393929f5/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.798341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, group_id=636f259ffddb4a96a0c83eb9393929f5, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.800252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:02.803941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 208/408] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:02.809811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:02.836315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 201/409] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 17 19:50:02 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/groups/636f259ffddb4a96a0c83eb9393929f5/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:02.955074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ad4f4-800b-4959-8fce-b144bdd9c15c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:02.994076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:02.994771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:02.995049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:02.995323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f/groups/636f259ffddb4a96a0c83eb9393929f5/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:02.997773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=80efcdefebdf4680bcd88df222732e4f, group_id=636f259ffddb4a96a0c83eb9393929f5, role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:02.999252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:03.030523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 209/410] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 17 19:50:02 2022] PUT /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f/groups/636f259ffddb4a96a0c83eb9393929f5/roles/1f37e1386b624e268925de2a48d79cc1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:03.093698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.209207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06c685-bd18-46f4-a689-b7d2ee6166b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:03.213414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:03.249487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:03.250565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.251048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.251516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.263981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cd91752b-a37a-476b-b038-54f4f27268ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:03.269792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:03.270704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:03.271513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:03.273565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 202/411] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:03.279740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.313133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:03.392548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c133-067f-47cd-aa72-0305b58571d3 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.393331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c133-067f-47cd-aa72-0305b58571d3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.393903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688c133-067f-47cd-aa72-0305b58571d3 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.396740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 203/412] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:03.399496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4a7e9-b10c-4597-bfbe-26c4e619f2f8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:03.401459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3118b-f352-46e6-8ad5-e93b141cc0e5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.401992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3118b-f352-46e6-8ad5-e93b141cc0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.402463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3118b-f352-46e6-8ad5-e93b141cc0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.438625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-04e3118b-f352-46e6-8ad5-e93b141cc0e5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:03.439859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-04e3118b-f352-46e6-8ad5-e93b141cc0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:03.442496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:03.443122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.443398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.443663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.444353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:03.444650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:03.446081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:03.462987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 210/413] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:03.487338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.592920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 204/414] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:03.646293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:03.664269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df126e-250d-412f-9a0a-a4ce740c777d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:03.703871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5904ca-267f-4e22-90a9-d73744559e5a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.704556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5904ca-267f-4e22-90a9-d73744559e5a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.705118 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5904ca-267f-4e22-90a9-d73744559e5a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.707294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 205/415] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:03.720271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.721033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.721732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.726373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:03.727049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:03.727333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:03.727595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] PATH_INFO: `/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:03.728911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] RBAC: Authorizing `identity:get_group(group_id=ca34b16edee14ec9be3aeafecc9ae48b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:03.730793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:03.733575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 211/416] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:03 2022] GET /identity/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:03.740103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.745562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:03.747005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:03.823092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 206/417] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:03.877072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:03.910110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:03.967609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83d232-3623-48ca-b2ca-e1ef98dca53e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.012874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.013525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.013836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.014102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] PATH_INFO: `/v3/projects/79e7870848b34ea38cb8567a241b0ce4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.016428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] RBAC: Authorizing `identity:get_project(project_id=79e7870848b34ea38cb8567a241b0ce4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.017927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.019610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 212/418] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:03 2022] GET /identity/v3/projects/79e7870848b34ea38cb8567a241b0ce4 => generated 344 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:04.026115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.040787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1be983-7f9f-4c03-9888-fc673780c2d5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.072282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.097312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.098003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.098320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.098589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.109130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:04.114365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:04.115265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:04.116458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 207/419] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:04.122702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.153026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5c344-32a3-48db-84df-bfb22ac1b8bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.208400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.209381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.209873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.210329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] PATH_INFO: `/v3/projects/79e7870848b34ea38cb8567a241b0ce4/groups/ca34b16edee14ec9be3aeafecc9ae48b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.213636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] RBAC: Authorizing `identity:create_grant(project_id=79e7870848b34ea38cb8567a241b0ce4, group_id=ca34b16edee14ec9be3aeafecc9ae48b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.216240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.235998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 213/420] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 17 19:50:04 2022] PUT /identity/v3/projects/79e7870848b34ea38cb8567a241b0ce4/groups/ca34b16edee14ec9be3aeafecc9ae48b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:04.236910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.242316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.262547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5dcd5-c19e-402a-98a3-a9a510f27fe8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.303937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.304978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.305503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.306171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.307196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:04.307700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.309679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.320691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 208/421] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:04.330166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.334708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.350586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2e338-6db8-4f21-82e7-735b87610585 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.388246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.388911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.389188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.389457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.402292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:04.407889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:04.409093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:04.410074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 214/422] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:04.416864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.558437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a7a1b-1065-4a99-add8-b335dcc3ac82 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.563526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c307f-8d9c-4461-a8bd-0c3737fb00d0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.610761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.611490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.611777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.612046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.612783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:04.613101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.614710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.628986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 215/423] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:04.630563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:04.646305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.647533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.648011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.648422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.671020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.674695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:04.807848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1e6e-e369-44cb-9b9f-eb9653e53c48 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:04.854444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.856005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.859139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:04.860551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:04.861256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:04.861750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:04.944929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:04.946802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:04.965945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.065030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 216/424] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:50:05.066640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.096409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 209/425] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:05.105267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.176907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85cf0b-10ab-4732-948d-e09bb2943d7e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.193999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.195736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.219809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.220849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.221344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.221831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.223228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.224924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.231470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 217/426] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:05.233478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.320725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a935-3062-42af-96cd-dc442358da34 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.343890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0f767-6457-47d0-9aec-63f8c502fde1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.381568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.382697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.383195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.383656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.385163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.386909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.392816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 218/427] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:05.395854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.442417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.443088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.443370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.443645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] PATH_INFO: `/v3/projects/0507e9f171104361a7ff74d326317861` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.444907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] RBAC: Authorizing `identity:get_project(project_id=0507e9f171104361a7ff74d326317861)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.446410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.448529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 210/428] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/projects/0507e9f171104361a7ff74d326317861 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:05.453851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.482005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.516566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743478ae-c8f5-4068-a40b-9e8c90f6edbe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.557566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.558584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.559110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.559853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.560825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=0507e9f171104361a7ff74d326317861) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:05.561339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.563297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca0b23-a494-4e93-b44e-26b0fca6f324 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.577599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9321654-d88b-4f69-88f9-e3df181eec45 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.578137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 219/429] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:05.579958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8027-5018-4183-8598-0e5961f4797c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.580446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8027-5018-4183-8598-0e5961f4797c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.581080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8027-5018-4183-8598-0e5961f4797c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.583204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 220/430] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:05.589530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38cf25-1a8a-4951-baed-b372640532c5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.590175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38cf25-1a8a-4951-baed-b372640532c5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.590638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38cf25-1a8a-4951-baed-b372640532c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.592325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.615294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.621737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.621737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.621737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.622785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.624589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61744fe7-b8c1-4460-8d5d-e93a6ba833bb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.631000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eb38cf25-1a8a-4951-baed-b372640532c5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:05.632299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eb38cf25-1a8a-4951-baed-b372640532c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.633663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 211/431] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:05.635668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6f299-c61c-4b0e-934e-46cec3a3396a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.636043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6f299-c61c-4b0e-934e-46cec3a3396a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.636385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6f299-c61c-4b0e-934e-46cec3a3396a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.638208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 212/432] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:05.639129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6f299-c61c-4b0e-934e-46cec3a3396a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:05.742176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 221/433] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:05.743241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.743524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.743868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.780044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:05.781268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:05.793303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6f299-c61c-4b0e-934e-46cec3a3396a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:05.819846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:05.835586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:05.836644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:05.837046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:05.837386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:05.838327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=0507e9f171104361a7ff74d326317861) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:05.838708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:05.840864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:05.848285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 213/434] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:50:05 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=0507e9f171104361a7ff74d326317861&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:05.874531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 222/435] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:06.027716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.084321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.147102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aea2ce-b02d-4395-acec-b7497bfacebc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.184497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.185145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.185425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.185730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.198445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:06.205114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:06.206085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:06.206900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 214/436] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:06.213839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.298662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.352970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2372ff-78a4-466b-b5b9-5236ae6f3914 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.398131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.399328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.400413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.400775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.402070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:06.402375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.403809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.416388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 223/437] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:06.422444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.460573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6a7b6-fa65-48e1-b459-7aa6e3b0242d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.467469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.468159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.508069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.508069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.508069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.508069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.526602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:06.533310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:06.534320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:06.535133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:06.535748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 215/438] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:06.546035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.563522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7780d3-93b5-4601-ae51-f98cc3c36fd3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.610684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.611621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.611897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.612316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.689574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84e815-6892-4d8d-824f-d7181b965cbd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.710053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.711693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53047-74cb-451b-a436-2b64658bd084 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.730959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.731708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.731993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.732262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.732962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:06.733262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:06.734907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:06.743291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 216/439] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:06.748941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.753294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.787359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:06.860602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 224/440] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:50:06.861924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f803cc-e2be-423a-8b5d-201f2ab5c792 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.862205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f803cc-e2be-423a-8b5d-201f2ab5c792 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.862466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f803cc-e2be-423a-8b5d-201f2ab5c792 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.864880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 225/441] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:06.866289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f803cc-e2be-423a-8b5d-201f2ab5c792 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:06.887679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d93c3-44d2-4e4b-b026-b43c73581b87 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:06.925099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:06.926091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:06.926534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:06.926973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:06.992898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f803cc-e2be-423a-8b5d-201f2ab5c792 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.017108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.018606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.023119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.029785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.030433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.030708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.031029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.032302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.033833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f4826-d9bb-4dae-99db-fe116d55eb71 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.035548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 226/442] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.036664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.036946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.037238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.051694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.056099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:07.057342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:07.128732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 217/443] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:50:07.128732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.129032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 227/444] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:07.130798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.240990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a02738-ec6e-475f-b308-73466c6ae109 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.255739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea076-1eb1-4c83-90f8-fddf29ae73c4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.271724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.280500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.281164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.281561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.281831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] PATH_INFO: `/v3/projects/80efcdefebdf4680bcd88df222732e4f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.282998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] RBAC: Authorizing `identity:get_project(project_id=80efcdefebdf4680bcd88df222732e4f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.284605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.286567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 228/445] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/projects/80efcdefebdf4680bcd88df222732e4f => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.291488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.292732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.300366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.301442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.301944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.302421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.304157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.306729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.314785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 218/446] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.316337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.422290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11980a13-c181-46ec-80b6-9ef1946d45cb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.424407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afefced-9b3a-44a6-871e-66085ce07187 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.461388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.462042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.462324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.462432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.462621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] PATH_INFO: `/v3/groups/636f259ffddb4a96a0c83eb9393929f5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.463293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.463747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.463831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] RBAC: Authorizing `identity:get_group(group_id=636f259ffddb4a96a0c83eb9393929f5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.464142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.464873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=8f4a64d3f31545619fe7a9e422d55398, scope.project.id=80efcdefebdf4680bcd88df222732e4f) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:07.465205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.465335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.466840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 229/447] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/groups/636f259ffddb4a96a0c83eb9393929f5 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.466990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.467760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.477107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 219/448] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=8f4a64d3f31545619fe7a9e422d55398 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:07.478023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.557486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.562962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.613207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5739c-a4b4-433e-b6bd-76965d755a9c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.616916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0198f-ae72-4ced-a988-a5cdd3db09b9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.662355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.662985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.665970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.665970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.665970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] RBAC: Adding query filter params (group.id=636f259ffddb4a96a0c83eb9393929f5, role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=80efcdefebdf4680bcd88df222732e4f) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:07.665970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.666285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.666285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.666285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.666285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.666540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.673046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 220/449] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/role_assignments?group.id=636f259ffddb4a96a0c83eb9393929f5&scope.project.id=80efcdefebdf4680bcd88df222732e4f&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.679155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:07.684277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:07.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:07.686221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 230/450] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:07.692396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.812138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f738460-6a55-4b43-a090-9cadd031cea5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:07.838390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:07.850730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:07.851357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:07.851637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:07.851912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:07.852617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:07.852930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:07.854403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:07.864149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 221/451] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:07.870532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:07.977639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747534db-9c0a-4e8e-b9c6-fe021458b61a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.003580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:08.015457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.016074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.016350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.016620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.085847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.087362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.188791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 231/452] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:08.194856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.299731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a46ce7-b990-4039-9dba-c5ce08f8b045 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.335523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.336157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.336432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.336696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] PATH_INFO: `/v3/projects/79e7870848b34ea38cb8567a241b0ce4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.337892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] RBAC: Authorizing `identity:get_project(project_id=79e7870848b34ea38cb8567a241b0ce4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.339332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.340939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 222/453] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:50:08 2022] GET /identity/v3/projects/79e7870848b34ea38cb8567a241b0ce4 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:08.346992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.374430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:08.453140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11c5db-6c2a-4e53-ae3b-f9c29ce52c38 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.493130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.494417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.494930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.495198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] PATH_INFO: `/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.496356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] RBAC: Authorizing `identity:get_group(group_id=ca34b16edee14ec9be3aeafecc9ae48b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.497845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be42f4-e133-4860-ba1b-fbd25bdbad8e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.499389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 232/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 17 19:50:08 2022] GET /identity/v3/groups/ca34b16edee14ec9be3aeafecc9ae48b => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:08.505795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:08.574976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:08.632506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c185a9a-b5eb-4c83-8451-1e1d257d0db4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:08.673015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:08.673720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:08.673913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:08.674210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:08.674943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] RBAC: Adding query filter params (group.id=ca34b16edee14ec9be3aeafecc9ae48b, role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=79e7870848b34ea38cb8567a241b0ce4) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:08.675247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:08.676692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c156af1-3a8e-4e63-b98c-1196a2fe5088 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:08.683666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 223/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 17 19:50:08 2022] GET /identity/v3/role_assignments?group.id=ca34b16edee14ec9be3aeafecc9ae48b&scope.project.id=79e7870848b34ea38cb8567a241b0ce4&role.id=c974a286ee1942c8bab82b232c6af142 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:10.958211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151932-d5b9-437d-867a-0d8515a03454 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:10.958799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151932-d5b9-437d-867a-0d8515a03454 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:10.958799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151932-d5b9-437d-867a-0d8515a03454 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:10.960367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 233/456] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:10.967034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525463-42db-47b2-9357-19608d72469f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:10.967964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525463-42db-47b2-9357-19608d72469f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:10.968435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4525463-42db-47b2-9357-19608d72469f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:10.998006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b4525463-42db-47b2-9357-19608d72469f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:10.999580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b4525463-42db-47b2-9357-19608d72469f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.088377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 224/457] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:11.198977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb186e0-ef0b-4d8f-8559-595a4f5b7be1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.199611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb186e0-ef0b-4d8f-8559-595a4f5b7be1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.200103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb186e0-ef0b-4d8f-8559-595a4f5b7be1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.201941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 234/458] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:11.207221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c3cc-0eb1-4bb0-860a-484d5b855a5b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.207772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c3cc-0eb1-4bb0-860a-484d5b855a5b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.208304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460c3cc-0eb1-4bb0-860a-484d5b855a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.232861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a460c3cc-0eb1-4bb0-860a-484d5b855a5b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:11.234496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a460c3cc-0eb1-4bb0-860a-484d5b855a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.289526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.322993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 225/459] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:11.432412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb186e0-ef0b-4d8f-8559-595a4f5b7be1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.534103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.543867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb186e0-ef0b-4d8f-8559-595a4f5b7be1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.574476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a65746-a7ce-4a0f-8b2a-b60a618f3b79 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.574476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a65746-a7ce-4a0f-8b2a-b60a618f3b79 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.574476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a65746-a7ce-4a0f-8b2a-b60a618f3b79 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.574476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 226/460] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:11.583128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.583412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.590399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.591159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.591446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.591712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.603291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:11.605263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:11.613752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.617248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:11.641561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:11.643035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:11.644145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:11.644145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:11.644145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:11.645947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 235/461] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:11.652264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.742548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 227/462] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:11.743784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.773879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312482e0-3ce6-4ae0-b734-6dfdc69802ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.843011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.843552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.843828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.844136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.844794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:11.845611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.847370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.872930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 236/463] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:11.880485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:11.887098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.906961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7444c5-e49b-4fc6-baab-202d760b6eb0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:11.917633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:11.945468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:11.946385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.946757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.947281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.959203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:11.960853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4cd114d6-8d4f-4802-98d2-61cb30a3b702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:11.977152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:11.978692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:11.979629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:11.979629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:11.979629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:11.983550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 228/464] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:11.984977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9dced-f587-439d-9a94-e52dd58f4359 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:11.985260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9dced-f587-439d-9a94-e52dd58f4359 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:11.985569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9dced-f587-439d-9a94-e52dd58f4359 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:11.991291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 229/465] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:11.992166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9dced-f587-439d-9a94-e52dd58f4359 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.016227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60df8cc-6532-494e-8f52-c5cd9926c955 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.058739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.059355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.059628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.059892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.060553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.062050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54649f22-ea43-432d-9d26-a18c1d4ed7e2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.096024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 237/466] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:11 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:12.096024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.096024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.096024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.110153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:12.111514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:12.140780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9dced-f587-439d-9a94-e52dd58f4359 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.204802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.210419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.211416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.211880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.212145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.212953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:12.213162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.214648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.224237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 238/467] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:12.225583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.242238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 230/468] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:12.250700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.295495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.365040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a85bdd-79f6-4781-9e8d-b7d3a0d1bc6c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.382546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf30610-340a-4991-9e34-e2c0de41b5d2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.403971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.404806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.405182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.405955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.419600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.421366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.421723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.422625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.423045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.423457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.424258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.425946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:12.442758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:12.443933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:12.445262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:12.445262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:12.445262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:12.445262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 239/469] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:12.450640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.454490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 231/470] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:12 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:12.510222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.562209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.577760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d0578-2f69-4548-8689-d6b974d74dc2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.614683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.615390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.615596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.615862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.616531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:12.616832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.618341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.631804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 240/471] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:12.637708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.672494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.715493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ea199-8a46-4c2f-80bf-71d932de3481 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.752646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9829de6-6c1b-4dbb-b9ea-04573b5a8583 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.754678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.757751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.758888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.759430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.759902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.773809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.775808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.791676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.792294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.792578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.792843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.793482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.795325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca41701-b23a-4a24-aed0-b664035a1ca0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:12.795730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:12.796634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:12.797990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:12.797990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:12.797990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:12.799927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 232/472] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:12.805653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:12.825195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 241/473] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:12 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:12.941120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6281705-0202-4803-a7ec-959d336cd988 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:12.972555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:12.983580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:12.984208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a56049-5a69-4bb1-9b9c-b45ead0d3a2f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.984467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a56049-5a69-4bb1-9b9c-b45ead0d3a2f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.984625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a56049-5a69-4bb1-9b9c-b45ead0d3a2f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.985448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.985999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.986378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 242/474] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:12.986731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:12.987746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:12.988271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:12.989964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:12.994072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:12.994353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:12.994632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:13.004891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 233/475] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:13.010239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:13.015660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:13.017726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:13.109582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 243/476] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:13.143057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742996d-8eb8-4dd1-8d11-867dcf628e99 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:13.180926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:13.181639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:13.181910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:13.182185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:13.182839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:13.184364 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:13.211316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 234/477] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:13 2022] POST /identity/v3/users => generated 249 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:13.412315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:13.501797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:13.516012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d95a0-a814-45bd-8dd5-0e7ae2c29945 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:13.552143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:13.552757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:13.553030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:13.553297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:13.554000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:13.555404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:13.564059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.sql.core [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:50:13.564059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 19:50:13.564059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [parameters: {'id': 'b275f7c3381f4e52aada80b8815f6441', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 19:50:13.564059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81533) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 17 19:50:13.581055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return ref.to_dict() Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:50:13.581978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application yield resource Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise self.value Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application session.commit() Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:50:13.582855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:50:13.583691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:50:13.584565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application result.read() Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:50:13.585363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [parameters: {'id': 'b275f7c3381f4e52aada80b8815f6441', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:13.586187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 17 19:50:13.586978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:13.589883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 244/478] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 17 19:50:13 2022] POST /identity/v3/roles => generated 144 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:13.590024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:13.646185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:13.707930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa068-6137-49fb-a938-d00fd664a0ca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:13.743394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:13.744061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:13.744330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:13.744604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:13.756986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ff122e18-a5b0-4735-8e53-8db58c015331 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:13.762205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:13.764380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:13.766328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 235/479] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:13.770093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:13.881795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dddeb-fed6-48ff-9019-1c8855d0b6b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:13.917612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:13.918237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:13.918513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:13.918778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:13.919793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:13.920094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:13.921569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6f20f-e415-4660-b274-b0e2265bc6aa None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:13.930281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 245/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:14.702066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8b2a-54d2-49e7-9f40-f23db778b412 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:14.702782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8b2a-54d2-49e7-9f40-f23db778b412 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:14.702782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a8b2a-54d2-49e7-9f40-f23db778b412 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:14.704332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 236/481] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:14.709689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06db6-3906-48d2-87f2-9752f906a601 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:14.710250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06db6-3906-48d2-87f2-9752f906a601 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:14.710250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06db6-3906-48d2-87f2-9752f906a601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:14.739786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-47c06db6-3906-48d2-87f2-9752f906a601 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:14.741022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-47c06db6-3906-48d2-87f2-9752f906a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:14.757572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:14.840812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 246/482] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:15.054808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2fefe-34fa-4a77-8a9d-e26e6f13c59d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.055220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2fefe-34fa-4a77-8a9d-e26e6f13c59d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.055385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2fefe-34fa-4a77-8a9d-e26e6f13c59d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.056838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 237/483] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:15.071521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ecdfb-169f-4e52-868f-ff24e8a39faf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.072021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ecdfb-169f-4e52-868f-ff24e8a39faf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.072178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ecdfb-169f-4e52-868f-ff24e8a39faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.096237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-486ecdfb-169f-4e52-868f-ff24e8a39faf None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:15.097569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-486ecdfb-169f-4e52-868f-ff24e8a39faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:15.199982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 247/484] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:15.219244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2fefe-34fa-4a77-8a9d-e26e6f13c59d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.287803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd68c59-c907-4adc-8369-fe6359031dcf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.288160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd68c59-c907-4adc-8369-fe6359031dcf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.288350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd68c59-c907-4adc-8369-fe6359031dcf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.296447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 248/485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:15.312120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.312550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.312749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.354949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2fefe-34fa-4a77-8a9d-e26e6f13c59d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:15.357963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:15.360103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:15.361370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:15.394076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:15.395068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.395533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.396017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.407553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:15.413163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:15.414179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:15.414986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:15.417989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 238/486] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:15.424231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.529366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 249/487] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 17 19:50:15.533729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.536113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:15.587738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26a3f-f9c9-4c94-9158-cdbd16212074 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:15.673067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:15.673734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.673921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.674388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.675242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:15.675690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:15.676343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a4230-ff45-45ca-9b86-f127058b14cb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:15.677268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:15.704236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 239/488] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:15.709658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.720954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:15.722492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.722958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.723414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.737846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:15.743227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:15.744311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:15.745110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:15.747014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 250/489] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:15.755048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.780720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:15.871179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:15.895854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673db85-494e-497f-a312-a15b0dbc9c07 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:15.926554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481bd4f-81bb-4a7b-bced-1e0ff801c0fa None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:15.936192 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:15.936821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.937095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.937363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.948801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:15.950317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:15.964977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:15.965943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:15.966954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:15.966954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:15.966954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:15.971085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 240/490] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:15.971956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:15.972435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:15.973211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:15.973636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:15.974019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:15.974928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:15.975355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:15.976972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:15.986379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 251/491] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:15.987478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.070185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.096913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785312e8-e0bd-4d61-86b2-10152176dfb0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.108277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80c528-e3c3-4db5-9883-81cdd76debee None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.134188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.134874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.135153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.135850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.143026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.147556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:16.153077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:16.154103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 241/492] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:16.155253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.159307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.159966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.160246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.160705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.161200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:16.161500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.162986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d68686-23f2-4307-ac98-95a2d3b0c6cf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.177629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 252/493] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:16.178919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17439c-8d42-449a-93a1-3658f77f0953 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.179205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17439c-8d42-449a-93a1-3658f77f0953 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.179471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17439c-8d42-449a-93a1-3658f77f0953 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.185568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 253/494] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:16.185568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17439c-8d42-449a-93a1-3658f77f0953 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.281421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bc9a5-c0b9-40ec-baa7-7c7b901375ad None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.294612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17439c-8d42-449a-93a1-3658f77f0953 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.320495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.321148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.321429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.321769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.330163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.334254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a556906-591d-4257-a283-c17a84476065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.334896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556906-591d-4257-a283-c17a84476065 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.334999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.335249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556906-591d-4257-a283-c17a84476065 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.335437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556906-591d-4257-a283-c17a84476065 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.336122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a556906-591d-4257-a283-c17a84476065 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:16.336421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a556906-591d-4257-a283-c17a84476065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.336526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.337906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a556906-591d-4257-a283-c17a84476065 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.347370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 254/495] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:16.348051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a556906-591d-4257-a283-c17a84476065 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f7b58a5b-d69b-4496-8d6d-7c3ed606c748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:16.358013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:16.359079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:16.359920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:16.359920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:16.359920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:16.361900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 242/496] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:16.364095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.364903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.365254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.387013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:16.388252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:16.410056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.467167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a556906-591d-4257-a283-c17a84476065 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.477590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 243/497] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:16.477590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.545625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.546526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.546798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.547062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.556402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.557963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:16.576545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:16.577482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:16.579066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 255/498] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:16.583278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.603235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad1dc-6279-4082-95f0-0e56e338a74c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.625110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.642259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.644013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.644522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.645291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.660516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.663271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:16.685437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:16.686623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:16.687516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:16.687516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:16.687516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:16.687516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 244/499] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:16.687516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.701914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970c238-46e1-4868-a592-c0228b850c2e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.714928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.743474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.744319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.744319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.744844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.745182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:16.745478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.747031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbeec6-a81a-4b2d-b770-6b25099a04c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.759761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 256/500] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:16.761017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42048-2f56-4c6e-b468-f0a96696c3eb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.761368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42048-2f56-4c6e-b468-f0a96696c3eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.761664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e42048-2f56-4c6e-b468-f0a96696c3eb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.762887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 257/501] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:16.763636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e42048-2f56-4c6e-b468-f0a96696c3eb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.804101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833dd83-623d-4b20-8aaf-686a835998e1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.845462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.846141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.846909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.846909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.847491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:16.847819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.849378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.875030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e42048-2f56-4c6e-b468-f0a96696c3eb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:16.889515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 245/502] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:16.890447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.914435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:16.915097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:16.915366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:16.915709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:16.916385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:16.916751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:16.918295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:16.929850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:16.930674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 258/503] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:16.931802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:16.964617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.016697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83cd47-417e-402b-b45e-ebaa866037d2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.042488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa921ef6-9b36-41d1-b5f3-cc4c2c8c09ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.051388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.052030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.052300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.052564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.063772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-87a53947-3b46-42d4-8ec8-3579f1096de4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:17.068516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:17.069526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:17.070383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 246/504] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:17.071568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.071568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.071784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.080676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.081311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.081609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.081888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.090187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.090477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:17.091671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.091791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:17.104175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:17.105184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 259/505] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:17.106581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.167633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 247/506] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:17.168643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.194463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.205476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.219432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f8dd6-449c-452b-ae12-96bdf5d5b9c0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.255757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.256373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.256646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.256910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.279201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545df023-8c1e-4c18-a547-88eaf73490ac None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.317908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.318551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.318829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.319101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.328721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.330269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.331725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.333462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:17.344014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:17.345027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:17.345980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 248/507] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:17.347309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.438591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.441496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.455751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 260/508] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:17.456868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.464551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff83219-e323-4986-97a1-823d252c5bca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.498677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.499315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.499589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.499855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.500520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:17.500822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.502539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.511465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 249/509] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:17.512302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.570167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce8a2b-b5c9-452e-8bfa-c297a7782183 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.604559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.605180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.605454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.605749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.606504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:17.606805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.608309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.614492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3abc41-9139-4bc1-80da-da9b4ad51b11 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.648280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 261/510] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:17.648280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.653522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.653522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.653716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.653963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.654842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:17.655012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.656405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.671813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.675831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 250/511] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:17.678465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.703539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.760584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882d0f-d1c8-468c-8326-c7a8afa78070 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.780075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239f0-629e-4769-b39e-36db55827b2a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.795926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.796653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.796926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.797189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.807873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.809353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.814982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.815621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.815899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.816174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:17.824604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:17.825876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:17.827124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:17.827124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:17.827124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:17.827124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 262/512] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:17.827124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.830641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.832183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:17.848008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:17.848977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:17.849896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:17.849896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:17.849896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:17.849896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 251/513] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:17.850126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:17.908409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.939102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:17.943450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5332-90da-4974-a5df-ca3dbafe34d1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.961821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9aa0e-31a7-4288-8ef5-437f05cf4aae None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:17.980452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.981097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.981374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.981693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:17.992803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:17.994358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:17.996425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:17.997057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:17.997405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:17.997713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:18.009602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:18.010806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:18.011686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:18.011686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.011686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:18.011686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 263/514] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:18.011907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.075678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.077103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.120405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f339-dd40-4494-a923-10afd36b62c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.131206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.158475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.159091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.159366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.159999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.181188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.184056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 252/515] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:18.184958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.232744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.234311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.287096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81151a3d-b52e-4196-bd6f-df337f258c26 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.323990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.324607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.324879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.325143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.325883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:18.326126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.327528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.340366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 253/516] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:18.340565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 264/517] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:18.341038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.341315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.360448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.392919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.446096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507e203-b729-4d2d-a895-9b495a9df7fd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.453072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285f003-b756-4163-a0e0-367e4fab5543 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.481287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.481926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.482199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.482468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.483115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:18.483411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.484825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.486467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.487085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.487366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.487642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.488294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:18.488599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.489992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.496621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 265/518] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:18.497947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.502271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 254/519] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:18.503220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.583079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.604335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baf79b-c945-4913-a416-d025e2bfccfc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.615432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.616143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1a84f-7d11-4267-8b96-bd49712c460d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.641097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.641838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.642111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.642422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.652321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.652932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.653205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.653470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.654142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.654390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.655576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.655858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:18.671430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:18.672553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:18.673438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:18.673438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.673438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:18.673438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 266/520] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:18.673712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.693601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 255/521] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:18 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:18.693601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.800953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba74df-c013-4149-8530-95dde1cf3d06 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.818647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d7b95-a75f-42d6-b964-60b2d164a93c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:18.835225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.848915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.849604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.849885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.850160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.860152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.861230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.867747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:18.867747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:18.868049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:18.868293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:18.875211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:18.876203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 256/522] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:18.877873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.882122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:18.883639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:18.900862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:18.902027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:18.902866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:18.902866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:18.902866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:18.902866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 267/523] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:18.902866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:18.924945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:18.997489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088ad42-19c8-4910-8ea5-455630cba5c0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.026479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ebd31-47da-432e-986e-9e7d770d6b4f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.035393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.036042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.036317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.036608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.045455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.047154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:19.060165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:19.061316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 257/524] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:19.062768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.068173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.068789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.069064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.069331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.070028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:19.070326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.071808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.088863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 268/525] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:19.088863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.122431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.194135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.198576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678928f-e7e9-4849-a880-da4e7a5ac400 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.223193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7995a-7414-4099-a77f-5da366a89d86 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.248752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.249473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.249790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.250064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.250807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:19.251115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.252564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.264537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.265441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.265941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.266379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.267233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:19.267729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.269135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.274777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 258/526] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:19.275648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.287048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 269/527] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:19.287900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.396971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58245bf-026b-47e0-bf05-77c340009cf9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.400280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafdca-1bec-4004-8652-91711fa557b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.405026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.440321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.441226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.441761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.442258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.443048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:19.443440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.444979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.445508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.446181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.446492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.447100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.449942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.456938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.458043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.464547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 259/528] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:19.465288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:19.471541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:19.472496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.473377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:19.475001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 270/529] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:19.480121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.584259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aed21-a3a9-4930-81a5-fb90d1e6b8a6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.596835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb8d0-2c37-4f84-bbe6-750144f4664e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.620570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.622074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.622432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.622778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.633119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.633780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.634052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.634379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.641463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.642959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:19.655593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:19.656632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 271/530] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:19.657488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.664781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.715475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.717407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.717570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.768255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea024b7-c9a1-43e8-af85-a14842ee15a8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.804651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.805250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:19.805526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:19.805814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:19.806587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:19.806889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:19.808356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:19.814126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 272/531] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:19.814692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.858135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 260/532] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:19.859059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:19.941092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.951607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ed01f-8742-4141-b03d-e79d9a087364 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.972633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:19.987500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fa665-79d2-42ef-afc1-3e0c3dc93ced None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:19.999140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:19.999761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.000039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.000307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.001560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.001887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.003804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.030006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 273/533] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:20.030878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.037004 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.037670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.037993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.038357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.039122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.039422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.040982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.058745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 261/534] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:20.059797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.144050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ebb8a-5e49-4a79-baf6-47d3b7e4d999 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.188548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.189186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.189458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.189779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.200847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5443b0-2aca-4168-97ab-eab9965d12b5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.202344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.203815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:20.219596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:20.220777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:20.221727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:20.221727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:20.221727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:20.221727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 274/535] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:20.221727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.235112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:20.244944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.245590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.245886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.246145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.246875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=9f03a7d5287c48e4821d308767f4c141) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.247179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.248611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.261737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 262/536] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/role_assignments?user.id=9f03a7d5287c48e4821d308767f4c141&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:20.262784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.263291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:20.422629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf319bb9-61b6-43c5-b024-346dd88c026f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.501648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488fbe6-56d3-46a2-9e8a-eee77f431bf6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.501872 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.501872 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.502068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.502436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.503185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=c97be7f3685b44cfa26aeae0f5b5d931) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.504448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.506017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.512785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 275/537] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/role_assignments?user.id=c97be7f3685b44cfa26aeae0f5b5d931&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:20.580426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.581220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.581634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.582001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.582756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.583149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.584630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01aa201-76d0-484f-b6a2-ee045ddb74ff None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.598347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 263/538] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:20.606917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.734964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276089-0845-461b-a026-7f60a3f9592a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.735459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276089-0845-461b-a026-7f60a3f9592a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.735739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54276089-0845-461b-a026-7f60a3f9592a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.737356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 264/539] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:20.743795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.744115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.744339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.749909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:20.767420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130828-d29c-4e52-850b-71f06211e34d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.772642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:20.773862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:20.813572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.814569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.814855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.815226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.824044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:20.825558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:20.838157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:20.839152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:20.840036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 276/540] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:20.846365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:20.859618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:20.935790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 265/541] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:20.961893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fade044-fcf9-48c0-b951-af27e39d4cd5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:20.996791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:20.997411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:20.997719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:20.997995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:20.998697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:20.999003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:21.000561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:21.017725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 277/542] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:21.019701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:21.136829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ac566-06e2-4345-bcbf-4e7c5d55df17 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:21.172348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:21.173190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:21.173652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:21.174080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:21.174968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=4eae163ceee54be9a10f7475ab87392e) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:21.175437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:21.177092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:21.182962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 266/543] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:21 2022] GET /identity/v3/role_assignments?user.id=4eae163ceee54be9a10f7475ab87392e&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:21.336954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:21.361111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:21.454342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56fbd0-d67c-4a45-a3e2-888d20576ee9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:21.493517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:21.494249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:21.494530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:21.494805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:21.505686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:21.510491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:21.511391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:21.512213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 278/544] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:21.518913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:21.581400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:21.659763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ca7db-2d03-4adc-9273-447cc2be070a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:21.699464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:21.700227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:21.700605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:21.700976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:21.701772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:21.702175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:21.703667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:21.717170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 267/545] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:21.722765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:21.806124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:21.833235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5b229-5fe0-4847-933b-89ec33f454d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:21.870040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:21.870680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:21.870958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:21.871229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:21.882155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:21.883752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:21.899896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:21.900927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:21.901818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:21.901818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:21.901818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:21.901818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 279/546] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:21.908507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.053244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14471019-de00-4314-9b7f-8dc2c4959796 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:22.065948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:22.107130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:22.107745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.108024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.108296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.108940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:22.109241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:22.110641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eedb4c-ee45-4ab1-bc27-1a08cb3e2861 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:22.133377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 268/547] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:22.139237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.230745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fcf5-fb8c-4e6e-842f-0e6d944b82d1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.231088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fcf5-fb8c-4e6e-842f-0e6d944b82d1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.231541 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5fcf5-fb8c-4e6e-842f-0e6d944b82d1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.234075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 269/548] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:22.239128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.239648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.240717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.262369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:22.264201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:22.291874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5141c-7522-44d6-9aa4-206502a57d04 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:22.347973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 270/549] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:22.353429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:22.378831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:22.379477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.379749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.380012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.394647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:22.396131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b8fa1ac2-4bf1-41b7-a77f-354475a11c63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:22.410823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:22.412291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:22.413410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 280/550] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:22.420062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.470209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885c26-6297-4f16-a8ff-4ad67ad0c882 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.470564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885c26-6297-4f16-a8ff-4ad67ad0c882 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.470755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885c26-6297-4f16-a8ff-4ad67ad0c882 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.472388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 281/551] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:22.477571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.477871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.478213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.502836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:22.504092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:22.595972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:22.608464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888d9ff-c523-4a96-bcf9-748488c40bde None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:22.641649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 282/552] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 17 19:50:22.664185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.670075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:22.670709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.670987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.671255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.671995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:22.672299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:22.673773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:22.705061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 271/553] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:22.708942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.844987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3164939-9ee2-4f52-93a3-74ff511507ca None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:22.874977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:22.887440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ccf3d-846a-446c-ad06-e4a9ae75fb96 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:22.909977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:22.910615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.910894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.911174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.930171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:22.935523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:22.936554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:22.937660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 283/554] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:22.945599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:22.964519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:22.965241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:22.965528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:22.965832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:22.994976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.040913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.042508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.051178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c089c-56be-4c91-b4fd-d38364d1088d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.086188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.086796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.087078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.087395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.088075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:23.088376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.089781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177a4f2-b86b-4e2f-808e-ee151605568c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 284/555] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8f2b-4b46-47ab-8450-5fa9318a2873 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8f2b-4b46-47ab-8450-5fa9318a2873 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8f2b-4b46-47ab-8450-5fa9318a2873 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 285/556] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:23.102463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8f2b-4b46-47ab-8450-5fa9318a2873 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.146035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.153988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 272/557] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:23.158517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.212434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8f2b-4b46-47ab-8450-5fa9318a2873 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.213078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.250860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.251501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.251774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.252040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.260020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d61d72-a2e4-45bf-bee2-2bf34790f5b1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.263769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-8bf691a0-22bb-4e13-aad2-92b5e544d599 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:23.268896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:23.269843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:23.270649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 286/558] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:23.271326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578884-a727-4379-b422-207c912e1036 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.271932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578884-a727-4379-b422-207c912e1036 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.271932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc578884-a727-4379-b422-207c912e1036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.289527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dc578884-a727-4379-b422-207c912e1036 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:23.290809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dc578884-a727-4379-b422-207c912e1036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:23.299923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.300566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.300854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.301334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.311264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.312758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:23.328957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:23.329951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:23.336069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:23.336069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.336069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:23.336069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 273/559] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:23.336069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.363303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 287/560] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:23.364189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc578884-a727-4379-b422-207c912e1036 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.393636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.448568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7c508-81c3-45aa-a116-2bb908fe081e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.455275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.475851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc578884-a727-4379-b422-207c912e1036 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.484850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.486150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.486631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.487363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.501773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.504057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.509744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.510357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.510633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.510902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.511574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:23.511872 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.513289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.521255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 288/561] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:23.522081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-01a1192f-9a12-4180-8452-930e1655a5a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:23.522647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:23.523586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:23.524484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:23.524484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.524484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:23.526461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 274/562] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:23.527779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538fc45-007f-46b4-9875-42842dd8298a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.528124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538fc45-007f-46b4-9875-42842dd8298a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.528458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538fc45-007f-46b4-9875-42842dd8298a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.530374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 275/563] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:23.530578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538fc45-007f-46b4-9875-42842dd8298a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.633400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.638838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538fc45-007f-46b4-9875-42842dd8298a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.643514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b389fd-0f0d-4dbf-a3ee-edde5e152fdb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.674664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.675302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.675579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.675846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.677365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.678004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.678279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.678550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.679194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:23.679493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.680882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.686583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.688035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.694048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 289/564] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:23.694857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c2004af2-dac8-44bd-a9aa-1788ecbc3411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:23.702626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:23.703601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:23.704503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:23.704503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:23.704503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:23.704503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 276/565] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:23.705043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.705137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.705406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.724125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:23.725331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:23.728178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.794496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 277/566] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:23.795560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.808935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ce373-0bd6-4c37-b25e-9a35e1e725f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.846882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.847631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.847906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.848168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.848809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:23.849103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:23.850618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:23.864673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 290/567] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:23.865769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.880300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.901140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f215-586e-4d2e-8590-761e57d3e223 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:23.935875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:23.936551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:23.936807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:23.937068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:23.948174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:23.951647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:23.957557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:23.958661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:23.959685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:23.962445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 278/568] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:23.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:23.989788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51e435-76f6-431d-ac90-4c553ba67bba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.026125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.026861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.027238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.027508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.035599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.037041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:24.049304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.050276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:24.051208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 291/569] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.051820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.070725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426e447-74ee-4a62-8726-c2bc83d5e224 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.103169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.103770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.104044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.104311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.104957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.105258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.106661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.117566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 279/570] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.118342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.122456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.152674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7a5dc-026e-489c-a2cc-0ff6a1ab93d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.178025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.187194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.187918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.188188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.188448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.195003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.196436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:24.207523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.208479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:24.209423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 292/571] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.209961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.222818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7723f32-363a-459e-8aaf-6f9646b68434 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.255785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.256390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.256663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.256931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.257625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.257927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.259342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.276972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 280/572] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.277932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.311879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555e276-8599-4975-9eaa-8489debda519 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.339083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.345555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.346939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.347224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.347497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.348228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.348533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.350009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.368480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 293/573] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.369403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.374540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905ac61-d04d-41f1-9802-d8fbe9242e97 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.397465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.409942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.410564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.410837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.411097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.421776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:24.428170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.429195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 281/574] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.434553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.473334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b56d8-c5a6-44e3-9b0b-9e4009cb720a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.507325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.507953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.508226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.508819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.516286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.517750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:24.529269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.530384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:24.531357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:24.531882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-0144-49d6-a870-2f56f3adedb4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.531965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 294/575] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.531965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.570819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.571429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.571729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.571996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.572685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.572985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.574473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.584323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.586687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 282/576] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.587538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.629433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.637613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a6118-3967-4e41-ae60-4a1c0204a2d0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.672442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.673067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.673338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.673632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.683884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.685349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.685461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a12719-3c9f-4531-b7a2-9f978d7c5274 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:24.699525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.700530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:24.706535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:24.706535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:24.706535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:24.706535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 295/577] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.706535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.718710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.719350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.719631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.719900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.729827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.731216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:24.745474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:24.746624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:24.747719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:24.747719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:24.747719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:24.748141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 283/578] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:24.749242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.796728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.799644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3a7b3-bd76-4126-a7f2-7d785faa5f17 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.834207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.834807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.835058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.835331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.836218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.836328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.837757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.845108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:24.846111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 296/579] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.846928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.853279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a32ccd-34bd-44b8-89c2-40ddafca28b5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.887604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.888511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.888796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.889060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.889858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.890160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.891601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.911631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 284/580] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.911631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:24.950138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430985-55fe-400b-815e-1b534c9ec2af None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:24.986193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:24.986728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:24.987003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:24.987278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:24.988002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=103c6c2fb3104914b8b4ec7c196f4996) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:24.988304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:24.989773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:24.995552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 297/581] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/role_assignments?user.id=103c6c2fb3104914b8b4ec7c196f4996&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:24.996329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.012399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca35ce-b5c6-4a78-9205-209bb6b298eb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.035169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.053180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.054021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.054348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.054677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.055414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.055761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.057268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.070120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 285/582] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.075562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.092955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.119075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8ea32-eecb-4a72-afff-85be79746728 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.159921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.160655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.160961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.161308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.162036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.162396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.164263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.178710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 298/583] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.179615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.193327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32b3cc-e0b0-4e3f-a2bd-8096526571ca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.228049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.228677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.228949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.229214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.239680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.241074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:25.255046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:25.256216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:25.257220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:25.257220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:25.257220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:25.257220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 286/584] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:25.261284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.270273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.287146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575443dc-a132-4852-94f7-d1116b6466c0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.325732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.326522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.326800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.326960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.327264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.339113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.340704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:25.355675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:25.356844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:25.357936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:25.357936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:25.357936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:25.363151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 299/585] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:25.363151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.374782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13608dde-f2c0-4b12-a4a3-9c5f49a67d6f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.411948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.412592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.412864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.413134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.421237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.422745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:25.436063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:25.437032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:25.437979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 287/586] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:25.438584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.478027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ce6e-fa92-4ec7-b064-f189833c535a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.515015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.515570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.515844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.516109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.523306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.524779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.526647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:25.538917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:25.539862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:25.540908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:25.545165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 300/587] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:25.545943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.547584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4efb16-9524-4111-a081-876444400e9c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.573153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.584182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.585353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.585862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.586411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.587475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.587776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.589139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.606021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 288/588] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.606765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.659714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7b3c-d0ed-4793-9362-cf3e95441c9c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.696964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.697988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.698414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.698680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.699321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.699614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.701018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.713014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21cdfd-bb3d-4096-ac5e-bd6f0e2aef04 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.713141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 301/589] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.713845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.763340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.764043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.764390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.764735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.765630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.765918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.767411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.788721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 289/590] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.788721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.789637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.831269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:25.838408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea2c3d-d998-4bcc-8125-3fa2c5b5136d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.874080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.874678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.874953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.875220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.875913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:25.876213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.877667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.893817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 302/591] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:25.893817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.896484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663fc-5b20-411b-ae4a-3c17ae294253 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:25.936551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:25.937183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:25.937453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:25.937743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:25.944915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:25.946374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:25.957848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:25.958837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:25.959765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 290/592] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:25.964344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:25.998327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039ce6e-9c95-4a73-ac18-646eb8fcc0b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.026222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.049327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.050069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.050351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.050620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.058076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.059568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:26.071180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:26.072107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:26.072979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 303/593] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:26.073464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.079611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7231596-c589-49f8-9961-103e36b74e65 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.081863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.117148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.117816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.118090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.118356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.129071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.130530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:26.144601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:26.145724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:26.146623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:26.146623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:26.146623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:26.146623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 291/594] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:26.146924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.185453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae65a-0c80-4e7d-8463-3cde35d8cad7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.221249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.221930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.222207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.222478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/c55245851594417e937d6b554d67ca31` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.236928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=1244993f093f4ad0920e3a42725a2308, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.238430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.251773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 304/595] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:26 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:26.252408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b6795-822a-4aa7-a776-2bfe8ec79377 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.253097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.303477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.305772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.306997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.307520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.308014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.308985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:26.309498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.311598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.340331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 292/596] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:26.341790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.359412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.405642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970e14-f76f-408a-b1fa-497521c7d8bd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.444522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.445293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.446569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.446569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.446846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:26.447480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.448851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.461684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 305/597] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:26.463013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.474749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdefa4-7223-42ed-b5d4-6891c464a333 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.512034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.513156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.513657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.514066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.514903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:26.515465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.517058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.529758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 293/598] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:26.530773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.578384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.597455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac9b2d-c22a-4058-bd02-2f383ec97ec9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.611708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.644458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.645343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.645642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ce991-bc52-4eea-8f99-e03a8a90cc4f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.645964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.646420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.663535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.665244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:26.680543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:26.681786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:26.682862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:26.682862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:26.682862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:26.689024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.689024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.689024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.689024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/9f03a7d5287c48e4821d308767f4c141/roles/c55245851594417e937d6b554d67ca31` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.689458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 306/599] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:26.690425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.705388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=9f03a7d5287c48e4821d308767f4c141, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.707095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.727171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 294/600] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:26 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/9f03a7d5287c48e4821d308767f4c141/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:26.728283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.804072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e68a19-3611-4609-881f-494bcad2d5cd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.848524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.849567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.850069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.850528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.858944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.860840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.862242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5890ea-2814-41e5-9641-0f79f7f8625b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:26.864524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e96a76a1-7911-406a-a79f-44348e849517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:26.876541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:26.877485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:26.878409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:26.880909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 307/601] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:26.887669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621b924-e2ab-412e-ae46-94f5f945be87 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.888016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621b924-e2ab-412e-ae46-94f5f945be87 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.888218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621b924-e2ab-412e-ae46-94f5f945be87 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.893985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 308/602] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:26.894934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621b924-e2ab-412e-ae46-94f5f945be87 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:26.899907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:26.921096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:26.922122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.922620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.923204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.924094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:26.924592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:26.926339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97321e65-8ff9-40ee-9388-0c387489b40d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:26.940335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 295/603] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:26.946430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:26.947112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:26.947579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:26.967809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:26.969349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:27.026350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621b924-e2ab-412e-ae46-94f5f945be87 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.043571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 296/604] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:27.044821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.076610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.077258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.077554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.077826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.078577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:27.079078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.080398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.097642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 309/605] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:27.101091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.162918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b102df7-4451-42fe-897b-bcdcdbb390c3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.177485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:27.185886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:27.216203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.216778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.217048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.221474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.234480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.235934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:27.257853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:27.258810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:27.259794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 297/606] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:27.266637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.270400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa054-13f5-4113-abde-ebfaecbba6b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.306232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.307246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.307748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.308218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/c97be7f3685b44cfa26aeae0f5b5d931/roles/c55245851594417e937d6b554d67ca31` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.324234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=c97be7f3685b44cfa26aeae0f5b5d931, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.326170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.350824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 310/607] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:27 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/c97be7f3685b44cfa26aeae0f5b5d931/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:27.415809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e162b-e806-4398-9a12-8e6620c67b5a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.441856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.467168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.467998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.469527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.471494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.473037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:27.473462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.475230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.504641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 298/608] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:27.511302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.559784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:27.609022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ab36-f3a7-42bc-b7b1-8231845c4715 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.625914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:27.639303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677f3f8-f6fc-4693-882c-4db80fd5d46e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.654049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.654691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.655119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.655387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.675196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:27.679790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:27.680949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:27.681953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 311/609] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:27.689927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:27.711796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.712501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.712778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.713050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/4eae163ceee54be9a10f7475ab87392e/roles/c55245851594417e937d6b554d67ca31` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.746122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=4eae163ceee54be9a10f7475ab87392e, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.747594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.772631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 299/610] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:27 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/4eae163ceee54be9a10f7475ab87392e/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:27.838210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1cbcd-16b3-43c3-a6dc-eabca7e1920e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:27.876775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:27.877389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:27.877702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:27.877968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:27.878644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:27.878989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:27.880411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:27.890010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 312/611] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:27.895607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.030631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:28.036317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f4038-a324-49ca-823b-7ed120e2d439 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:28.073246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:28.073922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.074195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.074463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.085316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:28.086852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:28.101736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:28.102733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:28.103703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:28.103703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:28.103703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:28.103703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 300/612] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:28.108901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.155735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:28.232825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625b3ec-da90-40f1-9adf-46a88d06966c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:28.267539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:28.268149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.268425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.268693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.269344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:28.269674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:28.271057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6ceed-fbbb-4445-ab52-272791fe0ab6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:28.283777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 313/613] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:28.290057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.437800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddabf51-00fb-4b99-89b6-da949d89a53f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:28.441369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad8ae-b99a-48d0-9d22-981fa2badfcf None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.441369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad8ae-b99a-48d0-9d22-981fa2badfcf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.441369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad8ae-b99a-48d0-9d22-981fa2badfcf None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.442992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 314/614] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:28.459034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.462579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.462579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.479049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:28.482197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:28.484030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:28.484853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.485275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.485717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.487918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:28.496428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:28.502478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:28.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:28.523207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:28.524209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:28.527947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 301/615] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:28.536478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.596148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 315/616] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:28.672417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3c0d5-ad2d-44b6-9104-7e85a1d5e908 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:28.709093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:28.709746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.710025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.710297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:28.711049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:28.711352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:28.712822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:28.737164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 302/617] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:28.742368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.794040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:28.911725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:28.915993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6583cb-5bb4-4171-83a9-0a1033ae936e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:28.982430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:28.983467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:28.983959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:28.984431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.001314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.003506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.005217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:29.027488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:29.028564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:29.030073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:29.030073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:29.030073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:29.030073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 316/618] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:29.035598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.072719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768e22f-a47f-4cc9-8e8c-af0db1b0976c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.108063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.109127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.109405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.109816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.120732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-959e9514-8612-4c29-973b-d2833d9a6720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:29.125711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:29.126652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:29.127497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:29.131877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 303/619] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:29.133195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6376e-8fb6-48e1-9278-6382c354964f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.133572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6376e-8fb6-48e1-9278-6382c354964f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.134123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6376e-8fb6-48e1-9278-6382c354964f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.135723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 304/620] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:29.139790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6376e-8fb6-48e1-9278-6382c354964f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.147178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.183178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf7fe9-f1e2-4e57-a5c8-fda380a7c8b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.230640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.232148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.232619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.233120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.234975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:29.235705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.238023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acc653-4eb6-478e-b53c-c10696fa1f5e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.247026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6376e-8fb6-48e1-9278-6382c354964f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.262109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 317/621] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:29.263752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.264273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.264544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.267367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.283138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:29.284336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:29.292526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.293798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.294736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.295245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.296287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:29.296780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.298569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.309172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 305/622] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:29.310392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.371206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 318/623] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:29.371206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.441374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe916ec6-e847-4dea-ad4a-b36a8a9f465f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.481269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.482324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.482926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.483391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.491221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.492918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.499923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:29.506360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:29.507546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:29.508500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:29.515598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 306/624] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:29.516828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.590555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996f790-3496-40ca-af5d-a70ea1a78be5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.595883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.636238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.637308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.638368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.638864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.651803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.653630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:29.671032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:29.672054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:29.672950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:29.672950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:29.672950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:29.674779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 319/625] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:29.685148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.740831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16c92-959c-49ba-9be8-3f9805712712 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.782568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.783596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.784083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.784675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.785707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:29.786203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.787828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.801083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 307/626] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:29.807233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.824973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c747bc-274c-45da-8fa2-4b3e9ad7e4bc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.846309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.867786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.868618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.869025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.869309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.869995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:29.870293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:29.871861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:29.888323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 320/627] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:29.889195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.916328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:29.923522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e9183-0b57-4170-8629-ed3cef04241b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:29.960491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:29.961276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:29.961573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:29.961848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:29.973610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:29.978818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:29.979762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:29.980591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 308/628] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:29.981399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:29.999227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fbe5a-7bea-4a91-96b9-748680c5bb2e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.040272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.041379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.041902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.042379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/103c6c2fb3104914b8b4ec7c196f4996/roles/c55245851594417e937d6b554d67ca31` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.059313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=103c6c2fb3104914b8b4ec7c196f4996, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.060972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3669eec-3592-49b8-aff5-1b071f19c43c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.075817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 321/629] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:29 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/103c6c2fb3104914b8b4ec7c196f4996/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:30.077280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd26ad4-e7b5-40c2-bf6b-ee1a09621adb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.077577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd26ad4-e7b5-40c2-bf6b-ee1a09621adb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.077851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd26ad4-e7b5-40c2-bf6b-ee1a09621adb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.079072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 322/630] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:30.079804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd26ad4-e7b5-40c2-bf6b-ee1a09621adb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.096748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f245b39-9cb5-4f95-8aea-4ef644ba5125 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.114288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.135514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.136171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.136450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.136721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.145530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.146977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a7491777-c54b-4fc3-864d-7da1c2e0b7f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:30.159227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:30.160340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:30.161344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 309/631] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:30.166585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0ae7-55bb-4344-b619-679d7534806e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.166872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0ae7-55bb-4344-b619-679d7534806e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.167146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0ae7-55bb-4344-b619-679d7534806e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.173396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 310/632] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:30.173396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b442a0-878b-4bc7-89ef-4e3b08bf4ffd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.173396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b442a0-878b-4bc7-89ef-4e3b08bf4ffd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.173396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b442a0-878b-4bc7-89ef-4e3b08bf4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.189963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-35b442a0-878b-4bc7-89ef-4e3b08bf4ffd None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:30.191235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-35b442a0-878b-4bc7-89ef-4e3b08bf4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:30.202083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.215217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd26ad4-e7b5-40c2-bf6b-ee1a09621adb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.257961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44001069-3449-43fb-84c4-c38652504861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.258598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44001069-3449-43fb-84c4-c38652504861 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.258835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44001069-3449-43fb-84c4-c38652504861 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.259151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44001069-3449-43fb-84c4-c38652504861 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.259838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44001069-3449-43fb-84c4-c38652504861 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:30.260133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44001069-3449-43fb-84c4-c38652504861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.261516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44001069-3449-43fb-84c4-c38652504861 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.269911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 323/633] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:30.270717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44001069-3449-43fb-84c4-c38652504861 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.282663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 311/634] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:30.284777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.285287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.285952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.309801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:30.311813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:30.363551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.377018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44001069-3449-43fb-84c4-c38652504861 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.402792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 312/635] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:30.404223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.413362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.414017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.414293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.414567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.415288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:30.415616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.417061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.435921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 324/636] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:30.441567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.525921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.575116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409129cc-9c31-4c30-a9ca-c83c10d463be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.616629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4a0c9-1eef-4e99-9b01-0a0df8a194e3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.618620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.620253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.620691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.621110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.634384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.636502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.654561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.655512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.655932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.656297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:30.656534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:30.657804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:30.658928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:30.658928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:30.658928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:30.660791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 313/637] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:30.661955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.675807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.730312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.731805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.774614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81523a-b30e-4e10-b6f9-84f14a9d376b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.801097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.810786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.811504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.811786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.812060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.822644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:30.827805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:30.828759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:30.829645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 314/638] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:30.830352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.846408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 325/639] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:50:30.847100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:30.898911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:30.940830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e3c94-0967-4f30-9d9b-c3bbc9e81f55 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.954041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86ab6f-763d-4d49-b4ed-c5927e4e016d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:30.979575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.980162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.980442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.980714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.981363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:30.981699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:30.983065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:30.991107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:30.991803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:30.992137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:30.992479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:30.996743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 315/640] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:30.997762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.008247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:31.013142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.014258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:31.015295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:31.016357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 326/641] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.017137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.055723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.111925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab50201-6aaa-4b1f-9b70-bc8b3971844d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.127578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc329995-ec3e-49b7-a45d-63d9bdb06b58 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.137550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.148400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.149015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.149297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.149595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.150381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:31.150688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.152045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.164645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 316/642] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:31.164645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.172154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.172480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.172751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.173019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.184434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.186391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:31.201261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.202631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:31.203768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:31.203768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.203768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:31.203768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 327/643] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.203768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.268282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0705ac-b627-4a36-a0f3-82b115104620 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.277484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.305941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.306707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.307058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.307400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.315783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.317225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.321158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce64aa-433e-43d3-9f7d-cd4b0b84c53c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:31.328455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.329521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 317/644] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.330691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.361566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.362450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.362934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.363267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.364017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:31.364386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.365859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.374341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 328/645] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:31.375151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.397198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.440726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8793b7f-16ba-4df9-b13b-dc2cda5ae8c0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.477456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.478232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.478605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.478986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.482105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99914605-f3ba-4ee0-9f1a-361212c9c7c7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.490321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.492246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:31.507229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.508382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:31.514431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:31.514431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.514431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:31.514431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 318/646] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.514431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.524655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.525377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.525735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.526060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.526777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:31.527143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.528585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.536225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.549834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 329/647] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:31.550762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.623264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e67cb-836f-4e1f-9efc-f092980f0399 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.661542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.662280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.662560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.662830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.663183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0ef89-dc81-4c39-a09a-b03ff7acaedd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.663560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:31.663865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.665331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.688061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 319/648] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:31.688061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.704081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.704785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.705123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.705453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.716550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.718317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:31.734227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.735369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:31.736474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:31.736474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:31.736474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:31.741726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 330/649] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.741726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.769232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.793239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e2880-80c9-4d45-a702-8fe98702519d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.829228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.829959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.830296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.830623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.831327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:31.831802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.833316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.841350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b334-2851-46b2-a515-5837d9e08d4e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:31.846975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 320/650] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:31.847795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.896489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:31.909952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:31.910608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:31.910889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:31.911166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:31.921329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:31.923493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:31.939259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:31.940345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:31.941440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:31.945437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 331/651] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:31.947310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:31.984205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed446017-78e7-4323-ae5e-38f0830f1773 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.028072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.028687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.029037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.029318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.038865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.066047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45334-fc0d-4bab-aef6-c3e4141d58d6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.099625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.100238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.100514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.100780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.101426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.101755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.103165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.103305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.104744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.115185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 332/652] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.116025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.172241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.221908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 321/653] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:32.222758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.226303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27365102-8633-4f80-95f9-7942c99efc62 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.260350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.260987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.261262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.261528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.265786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.268742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.270224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-09900f27-78b0-4a8b-8bae-d8b15bde622a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:32.281816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:32.282759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:32.283689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 333/654] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:32.284295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c195f-071d-4719-81e4-7ea00b515a12 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.284295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c195f-071d-4719-81e4-7ea00b515a12 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.284680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c195f-071d-4719-81e4-7ea00b515a12 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.285729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 334/655] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:32.286417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c195f-071d-4719-81e4-7ea00b515a12 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.328304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a387f-fac7-438e-8a40-032447f505bd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.360282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.360885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.361158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.361422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.362135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.362434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.363848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.375386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 322/656] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.376278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.387028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c195f-071d-4719-81e4-7ea00b515a12 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.399936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.420923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.421576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.421854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.422119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.429952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.431410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:32.442596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:32.443614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 335/657] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:32.444554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.481275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.485432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2bdb5-689c-4bd6-aab5-1b95d8d9edf1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.519714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.520415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.520688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.520963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.532147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.533561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.545335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417357-4830-4cc7-94f0-d5977d515c5f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0a612cf6-9c98-48ac-933f-571c8cac2fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:32.549935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:32.550871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:32.551760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:32.551760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:32.551760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:32.551760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 323/658] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:32.552409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.552523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.552776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.571693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:32.572931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:32.579883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.580492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.580766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.581030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.581788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.582091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.583537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.600391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 336/659] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.601182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.631942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.638965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 324/660] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:32.639806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.705545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a128b8-bf40-46bd-ba36-116497fbad03 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.722745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.741400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.742046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.742249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10160354-d05e-4b54-ba6e-fe65098f5257 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.742354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.742571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.743295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.743855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.745415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.757880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 337/661] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.760398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.787198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.787800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.788075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.788351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.789113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.789420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.790973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.813431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 325/662] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.814510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.873332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:32.881740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede277e-b909-4bda-9800-e178f0642614 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.923470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.924086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.924362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.925814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.925814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:32.925814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:32.927206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:32.934730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62491ea8-e4c7-4422-8080-99704dbc5e9e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:32.939369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 338/663] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:32.940501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:32.980608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:32.981345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:32.981650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:32.981915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:32.995514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.052573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a776da-1ac2-49e4-9211-d86f9d511836 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.069887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.071658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.090284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.090895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.091171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.091437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.117225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.164249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.165835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.177260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 326/664] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:33.178296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.221305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.279200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 339/665] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:33.279200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.289936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40fb0-ea7b-485d-bae6-eb9f29a65c65 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.329209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.329873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.330063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.330751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.333688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.344155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:33.350589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:33.351712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:33.352584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 327/666] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:33.353280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.382876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58848527-ea7d-42ea-9353-a7d82871db22 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.422751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.423678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.424087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.424470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.431750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.433347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:33.445039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:33.446179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 340/667] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:33.447113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.466592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.472105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b941250-b782-4ed1-92c8-aebbcd355616 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.511399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.512033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.512312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.512729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.524206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.525812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:33.541609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:33.542769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:33.543872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:33.543872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:33.543872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:33.545155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 328/668] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:33.546075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.556397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee461-7cd0-48c9-9d85-154f11d833ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.575474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.598196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.598833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.599111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.599379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.609729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.611189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:33.625432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:33.626416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:33.627324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:33.627324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:33.627324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:33.627324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 341/669] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:33.627972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.656778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41ba39-949f-4c70-b2dd-82536c83155a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.693078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.693749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.694024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.694351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.694946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:33.695240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.696749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.704667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 329/670] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:33.705450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.721652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.743412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c568-1cb7-46af-a395-b3f7f7809bae None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.779007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.779934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.780382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.780840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.781867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:33.782341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.784062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.797407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 342/671] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:33.798684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.812429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0cfc4-015b-421e-b7db-159f0f896a7d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.830508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.847168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.847790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.848076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.848404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.849031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:33.849335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.850780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.863513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 330/672] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:33.868700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.911373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa2e36-984f-4c79-8fe0-34970daf4424 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:33.943844 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:33.949267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:33.949986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:33.950264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:33.950580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:33.951241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:33.951580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:33.952926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:33.969955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 343/673] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:33.970781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:33.975027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554833f-336b-443d-a342-6199f7abeac5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.013195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.014264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.014538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.014813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.025342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.026832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:34.042288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:34.043276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:34.044561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:34.044561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:34.044561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:34.044561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 331/674] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:34.044561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.066316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.084112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969169f6-ef84-4130-b2f5-24f5747f0ac7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.117979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.118924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.119389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.119844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.120644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=9f03a7d5287c48e4821d308767f4c141) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.121008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.122536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.137583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 344/675] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:33 2022] GET /identity/v3/role_assignments?user.id=9f03a7d5287c48e4821d308767f4c141&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.137583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.171660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaba33-5c62-4914-8dd9-6ac1219fedb8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.203441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.222311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.223540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.223826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.224101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.233702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.235380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.250026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae529f5-ce93-4b08-a80c-0e51b373e91f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:34.251793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:34.252789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:34.253756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:34.255715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 332/676] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:34.256771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.297556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.298205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.298482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.298741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.308234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.309921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:34.324721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:34.325893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:34.326919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:34.328643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 345/677] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:34.329622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.344573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.394141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bc4bf-ae10-4798-8cfd-c5693097e64f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.432583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.433572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.434074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.434529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.435255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.435627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.437078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.446279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc004a-3d8b-4846-960c-6d4b9fd8b3df None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.450682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 333/678] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.451511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.486197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.487091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.487368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.487634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.488350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.488653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.491320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.503457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.506914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 346/679] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.507810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.564260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89383d8e-cd49-493c-9cc6-b981d67142fb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.599573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.600198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.600486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.600824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.601444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.601777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.603289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.615634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 334/680] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.615990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.616494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.630109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8714b-c0fd-4d69-a1dd-4d8e9374058d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.669661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.670529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.670954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.671365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.679638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.681670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:34.694239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:34.695512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:34.696658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:34.703932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 347/681] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:34.705189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.719412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbd87-6b1c-414c-84c8-856f558f0602 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.721756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.754178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.754889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.755232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.755568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.756370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=c97be7f3685b44cfa26aeae0f5b5d931) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.756745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.758286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.764504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 335/682] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/role_assignments?user.id=c97be7f3685b44cfa26aeae0f5b5d931&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.765313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:34.828051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc462cb6-b53f-4fe2-a26b-90b0ff370fbc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.868503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:34.879153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.880252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.880782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.881307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.883501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=4eae163ceee54be9a10f7475ab87392e) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.884225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.887476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa94594-d06c-45ee-a502-c4851551d40e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:34.902028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 348/683] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/role_assignments?user.id=4eae163ceee54be9a10f7475ab87392e&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:34.905448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3b8d9-1975-4385-a0ec-33aa1409c85f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.906115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3b8d9-1975-4385-a0ec-33aa1409c85f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.906643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3b8d9-1975-4385-a0ec-33aa1409c85f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.910478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 349/684] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:34.915498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b783cb-f7d7-411a-a1c7-b469098b276e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:34.919097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.919821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.919992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.946166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:34.947902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:34.973350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:34.974040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:34.974324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:34.974591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:34.975365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:34.975671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:34.977982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:35.009566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 336/685] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:35.011263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.040008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 350/686] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:35.061989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:35.132890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beedea-8da7-4a41-9c76-dbe5391708ec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:35.176878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:35.177417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:35.177751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:35.178023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:35.252855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:35.254707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:35.287875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:35.364954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 337/687] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 17 19:50:35.374561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.374718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.523427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709a004-8647-4187-80ed-29375f11d1a2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:35.552132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:35.563666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769e0d-054c-4cbc-8e13-49b935cfb2e4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:35.566126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:35.567129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:35.567612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:35.568083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:35.580926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:35.587795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:35.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:35.589548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:35.590202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 351/688] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:35.601145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:35.602552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.611634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:35.612277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:35.612553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:35.612822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:35.623765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:35.625214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:35.640191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:35.641164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:35.642104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:35.642104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:35.642104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:35.642104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 338/689] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:35.650900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.794709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192eafa-4f91-4280-89fd-3811b54192cb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:35.841562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0340e4-4f99-45da-a511-6a149091b977 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:35.874314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:35.875255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:35.875534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:35.875801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:35.876455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:35.876757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:35.878168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:35.880662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:35.881276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:35.881559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:35.881834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:35.882511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:35.882865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:35.884265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:35.902813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 352/690] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:35.905177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 339/691] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:35.906240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.910582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:35.986803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:35.990001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:36.087884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49cac-f327-4b9c-9419-dd4b1382b63a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.092999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e70ea0-a118-4fdd-bc6f-8e1b70719419 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.129758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.130009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.130542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.130881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.131216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.131378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.131857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.132290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.138669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.140201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.145665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.147685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:36.152247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:36.153112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:36.154047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:36.157352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 340/692] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:36.159698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:36.166944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:36.167875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:36.168776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:36.168776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:36.168776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:36.171727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 353/693] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:36.177200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.336632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:36.342180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:36.342303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db17df9-3e07-417b-bffa-ef5e0dfd9903 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.345035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46abcf-119f-407c-8272-1ed23ab9266a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.375682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.376410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.376753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.377087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.377848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:36.378213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.379659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d9a05-b86e-4a5e-9688-300636332d28 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.390636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.391252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.391532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.391805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.392547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:36.392868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.393066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 354/694] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:36.394303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bccc3-8dcc-48de-a6c5-5e8370d1a4e5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.394408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99d288-99d3-4551-8b3f-69303c0cbc80 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.394580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bccc3-8dcc-48de-a6c5-5e8370d1a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.394841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bccc3-8dcc-48de-a6c5-5e8370d1a4e5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.400589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 355/695] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:36.401371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bccc3-8dcc-48de-a6c5-5e8370d1a4e5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.411474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 341/696] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:36.412872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.413162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.413632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.434522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:36.435937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:36.510155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bccc3-8dcc-48de-a6c5-5e8370d1a4e5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.517864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 342/697] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:36.518806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.545842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.546548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.546826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.547094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.554889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.556544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:36.568566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:36.569557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:36.570511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 356/698] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:36.575822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.617930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:36.687975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:36.713163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84424f-e37a-4939-b775-1808122339f2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.758045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eb137-3f50-4f1c-b8d8-3727925d1f92 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:36.795089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.795703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.795976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.796243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.796979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:36.797877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.799803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.802985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:36.803599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:36.803875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:36.804141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:36.804868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=103c6c2fb3104914b8b4ec7c196f4996) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:36.805172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:36.806637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:36.826378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 343/699] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/role_assignments?user.id=103c6c2fb3104914b8b4ec7c196f4996&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:36.830018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 357/700] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:36.835986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:36.868606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.028974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.031049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea765f-9f50-4494-a0ba-6d8dcaba33c3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.060297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba7d7f-2029-43bc-9758-50a63f255eb9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.073323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.074023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.074611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.074888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.102780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.103879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.104195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.104486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.104777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.118074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.119674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2aebc797-ec71-42e3-b8ba-5c3fe33b4ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:37.135167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:37.136153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:37.137055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:37.137055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:37.137055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:37.137055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 358/701] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:37.137755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0afe-a756-4cbe-a6ae-d9daf328ef90 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.137947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0afe-a756-4cbe-a6ae-d9daf328ef90 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.138126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af0afe-a756-4cbe-a6ae-d9daf328ef90 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.139362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 359/702] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:37.145856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.146278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.146534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.158342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.159787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.168053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:37.169627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:37.248241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 360/703] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:37.249161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.280707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 344/704] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:37.286167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.349132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.359273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.409091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34a673-e3a3-4dda-900f-0f00cdd96861 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.414867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2ce0c-112d-400d-9c4d-d5329ed93cb0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.447973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.448655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.448934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.449203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.450115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:37.450417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.452520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af639dc-f811-406e-9a44-06c2a22aff18 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.459690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.460700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.461179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.461683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.466953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 361/705] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:37.468310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8145e-d370-4bea-9450-0eedecb50aa9 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.468600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8145e-d370-4bea-9450-0eedecb50aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.468869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8145e-d370-4bea-9450-0eedecb50aa9 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.474918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 362/706] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:37.474918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8145e-d370-4bea-9450-0eedecb50aa9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.478740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.480385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dba0c5c6-104f-471b-9fa6-4dc1acbbbedd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:37.495876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:37.496902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:37.499478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:37.499478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:37.499478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:37.499478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 345/707] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:37.500545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265daca-13f0-4214-8566-8188400f5800 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.500824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265daca-13f0-4214-8566-8188400f5800 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.501097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265daca-13f0-4214-8566-8188400f5800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.532479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9265daca-13f0-4214-8566-8188400f5800 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:37.533726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9265daca-13f0-4214-8566-8188400f5800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:37.598170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8145e-d370-4bea-9450-0eedecb50aa9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.613424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 346/708] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:37.614383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265daca-13f0-4214-8566-8188400f5800 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.622679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.646655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.647926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.648809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.649547 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.650506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.652052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.654074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.679024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 363/709] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:50:37 2022] POST /identity/v3/users => generated 254 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:37.679896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.753565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265daca-13f0-4214-8566-8188400f5800 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.804132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f5aa9-dd38-4984-96b7-b43ff38b1f9d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:37.843262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216397-45e4-45c0-8944-05251350d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.844026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216397-45e4-45c0-8944-05251350d670 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.844302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216397-45e4-45c0-8944-05251350d670 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.844568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56216397-45e4-45c0-8944-05251350d670 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.845231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216397-45e4-45c0-8944-05251350d670 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.846768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56216397-45e4-45c0-8944-05251350d670 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-56216397-45e4-45c0-8944-05251350d670 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:37.859180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:37.860295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:37.861161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 364/710] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:37.867463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:37.868101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:37.868437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:37.868738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:37.869080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216397-45e4-45c0-8944-05251350d670 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.869396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:37.869722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:37.871095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:37.882964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 347/711] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:37.888338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:37.914243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:37.984331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56216397-45e4-45c0-8944-05251350d670 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.004712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:38.015566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83efd36-4c4e-4919-9ba7-c1918f11b565 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.035450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.036131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.036573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.036779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.037482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.037901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.039347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.044834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 365/712] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:38.045724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.066102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.067140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.067521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.068339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.078690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.081311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:38.096790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:38.097980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:38.099063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:38.101563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 348/713] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:38.101868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.178359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb75b9e-7d34-4a45-a826-e1beb23279e9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.180185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:38.216482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.217226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.217589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.217926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.218580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.220023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:38.229193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:38.231403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:38.233424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 366/714] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:38.235480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.247943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6990f9b-918e-42d2-9d08-076c22918949 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.296518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.297153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.297717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.297997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.299193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.299617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.302139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.321722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 349/715] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:38.322925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.368011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:38.374665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043944bb-4167-435d-9ff0-e7e13e0d45e8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.445923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.446667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.446935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.447197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.447897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.448189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.449672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.475775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 367/716] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:38.483770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.530538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:38.530876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b0850-4978-4497-937b-2fc162beb108 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.582816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.583442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.583746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.584229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.584881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.585180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.587654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.594081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 350/717] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:38.599888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:38.634002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5ce85-f438-4c9b-9b59-8b97760fbd57 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.677043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.678364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.678737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.679070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.679859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.680224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.681819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f99362-b974-4277-995a-3bb1aa95feea None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.688676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 368/718] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:38.753446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:38.783705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa68b5-299f-461c-a420-9151fb1cac09 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:38.823274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:38.823894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:38.824172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:38.824441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:38.825085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] RBAC: Adding query filter params (type=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:38.825391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:38.826788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77e593-32d0-4629-ab80-7f4555dbed7d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:38.832158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 351/719] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:39.142496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65920eee-9d8a-4ca2-ae81-dbe737ba582d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:39.143050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65920eee-9d8a-4ca2-ae81-dbe737ba582d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:39.143459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65920eee-9d8a-4ca2-ae81-dbe737ba582d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:39.145187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 369/720] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:39.150227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:39.150741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:39.151224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:39.170568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:39.172092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:39.245029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 352/721] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:39.651149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65920eee-9d8a-4ca2-ae81-dbe737ba582d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:39.775275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:39.786735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65920eee-9d8a-4ca2-ae81-dbe737ba582d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:39.833636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:39.834206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:39.834482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:39.834751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:39.848073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:39.852810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:39.853955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:39.854809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 370/722] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:39.860101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:39.954252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:39.990916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94e776-7c16-4cdd-af66-7dacfe73b009 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.039548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.040637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.041146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.041628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.042488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:40.042981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.044897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae412e-001f-4715-b820-0724cc378ec5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.057405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 353/723] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:40.062917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.122672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce5bf-1a03-468b-9ea3-99457ab7afe0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.123522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce5bf-1a03-468b-9ea3-99457ab7afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.123993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce5bf-1a03-468b-9ea3-99457ab7afe0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.125835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 354/724] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:40.130539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.131075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.132037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.154055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:40.156330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.256900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 355/725] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:40.261337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cf572-408e-4f58-b137-dd4b5d439b66 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.319592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.320240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.320517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.320784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.339420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.340901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:40.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:40.363655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:40.364539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:40.364539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:40.364539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:40.364539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 371/726] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:40.369921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.461906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:40.505257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4570d09-73e2-49e6-bf98-69ea47513bc4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.559782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.560412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.560702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.560961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.561644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:40.561976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.563358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.581599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 356/727] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:40.589751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.592777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:40.624591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.757774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a47d-0dac-486f-80d8-79192e145ec1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.792572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.793243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.793520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.793813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.794055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45a7b96-b319-46f7-88a2-7a4034960820 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.803094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.804559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-98d5d39c-622a-4db5-b41d-ad89f8abd20d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:40.816154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:40.817116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:40.818083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 372/728] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:40.819165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705774d6-cf09-487a-be51-e9650c3d9f34 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.819165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705774d6-cf09-487a-be51-e9650c3d9f34 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.819315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705774d6-cf09-487a-be51-e9650c3d9f34 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.820790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 373/729] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:40.823921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705774d6-cf09-487a-be51-e9650c3d9f34 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.839936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.840593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.840872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.841141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.852308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-567537fa-5f71-449f-b3df-062f2b7743da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:40.858055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:40.858997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:40.861148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 357/730] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:40.862564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.862940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.863330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.885340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:40.886819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:40.897409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:40.941138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705774d6-cf09-487a-be51-e9650c3d9f34 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:40.958628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 358/731] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:40.959619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:40.990858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:40.991916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:40.992368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:40.992796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:40.993759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:40.994204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:40.996038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3145622-c07b-45b4-8e53-42795048b339 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.019665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 374/732] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:41.021330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f471e-913a-45f3-a49f-af743d761545 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.021763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f471e-913a-45f3-a49f-af743d761545 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.022047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f471e-913a-45f3-a49f-af743d761545 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.027677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 375/733] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:41.027677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f471e-913a-45f3-a49f-af743d761545 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.096207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d1fd3-a485-4caa-ba65-5cfda22946f9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.114701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.133912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.134834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.135263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.135681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.136486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:41.136933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.138478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda63f85-98e4-4c7b-beec-42fcbe4ec463 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.147512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 359/734] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:41.148895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.149325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.149825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.153341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f471e-913a-45f3-a49f-af743d761545 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.168308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:41.170086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:41.176471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.193241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.194044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.194489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.194838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.243927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 360/735] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:41.245870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.266701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.268311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.355478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76b028-97f1-4c18-842b-8d6ceab86061 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.367268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.381275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 376/736] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 17 19:50:41.382084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.396839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.397502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.397829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.398106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.408085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.409054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.410778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d8bcaf3c-31e5-4290-9438-ff7c82fcf78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:41.426517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:41.427546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:41.428559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:41.428559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:41.428559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:41.432615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 361/737] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:41.433781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9cc35-ab54-447d-982a-1718ffb3e116 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.434065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9cc35-ab54-447d-982a-1718ffb3e116 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.434333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9cc35-ab54-447d-982a-1718ffb3e116 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.439801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 362/738] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:41.440568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9cc35-ab54-447d-982a-1718ffb3e116 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.499522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873ba1-d336-41ed-9047-b0666482512c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.533184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.533822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.534101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.534368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.535024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.536419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.548560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9cc35-ab54-447d-982a-1718ffb3e116 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.553624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 377/739] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:41.554440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.604812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.605588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.605796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.606068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.617823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.619474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-da9a6122-f0c4-4e71-aa29-2200b9ede7f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:41.635905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:41.636994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:41.637946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:41.637946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:41.637946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:41.637946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 363/740] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:41.639393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.639733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.640079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.659515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:41.660744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:41.663668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.665655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.702800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711a375-9fa4-46fc-8946-b0aa7dcec8a7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.742105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.742725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.743047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.743333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.744067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:41.744359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.745855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.762642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 378/741] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:41.763257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 364/742] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 17 19:50:41.763340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.763432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.882715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3e86c-356a-43a4-9bdd-d5f325d6a89d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.883186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f9c10-0666-4650-a4e1-2cc0e4be4704 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:41.922356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.922996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.923273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.923539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.927478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:41.928313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:41.929041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:41.929715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:41.930827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:41.931368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:41.932070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.934567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:41.935296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:41.941034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:41.941998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 379/743] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:41.943486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:41.949646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:41.959203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 365/744] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:41.960752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.054292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94084db1-9c13-4267-8aae-452846aaa463 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.081544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95edd325-86d1-447f-843d-87c02ac68cbb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.089225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.089894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.090172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.090443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.098185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.099657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:42.113335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:42.114372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:42.115321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 380/745] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:42.116568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.118568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.119502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.119788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.120058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.120734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:42.121036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.122449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.135420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 366/746] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:42.135420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.178109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.196299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.220552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cbda7-d7a1-416e-815f-104bfc84dd03 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.238461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba5d71-1c01-46c8-ac75-216e11c22218 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.254760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.255392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.255666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.256102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.262730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.264182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.272911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.273524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.273830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.274101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.274790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:42.275419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:42.276278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:42.277641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:42.278239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.278311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 381/747] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:42.278311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.300349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 367/748] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:50:42 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:42.301754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.401273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a73acf-1f20-446b-a684-4e37e23cdae2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.425786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.428237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a37c-d22d-4781-aed7-c620863ebfcd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.443411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.444503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.445002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.445615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.446676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:42.447192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.448930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.449419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.469755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697233-6096-4e40-b784-1a497023037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.470394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697233-6096-4e40-b784-1a497023037b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.470672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697233-6096-4e40-b784-1a497023037b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.470939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c697233-6096-4e40-b784-1a497023037b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.473053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 382/749] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:42.474047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.483758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697233-6096-4e40-b784-1a497023037b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.485204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c697233-6096-4e40-b784-1a497023037b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4c697233-6096-4e40-b784-1a497023037b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:42.501253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:42.502434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:42.503550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:42.503550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:42.503550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:42.503550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 368/750] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:42.504364 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697233-6096-4e40-b784-1a497023037b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.601523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c4f23-4a76-47f5-b051-c9c1fb1174ab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.645059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c697233-6096-4e40-b784-1a497023037b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.652156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.653142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.653656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.654145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.655090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:42.656372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.658744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.677016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 383/751] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:42.678233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.689380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.690051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.690333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.690600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.733349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.733609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.767144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.768969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.809150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e4853-9ae0-4a8d-8688-7498a78d495e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:42.855619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:42.856391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:42.856767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:42.857148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:42.858103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:42.858502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:42.860649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:42.876950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 384/752] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:42.883395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.884151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 369/753] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:50:42.885676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:42.976769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.989752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:42.994990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c32962-9235-4e99-9d1d-dd0fb6e37535 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.019896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1d8ab-06b8-4b62-8965-195dfc9ea62d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.033281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.034258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.034695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.035105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.036018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=103c6c2fb3104914b8b4ec7c196f4996) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:43.036500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.038425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.050012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 385/754] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/role_assignments?user.id=103c6c2fb3104914b8b4ec7c196f4996&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:43.051388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.065090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745722fe-1525-4d36-b181-28953709d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.065891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745722fe-1525-4d36-b181-28953709d80e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.066239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745722fe-1525-4d36-b181-28953709d80e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.066572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745722fe-1525-4d36-b181-28953709d80e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.075708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745722fe-1525-4d36-b181-28953709d80e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.077998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745722fe-1525-4d36-b181-28953709d80e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-745722fe-1525-4d36-b181-28953709d80e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:43.092875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:43.093998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:43.094947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 370/755] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:43.100497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745722fe-1525-4d36-b181-28953709d80e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.223563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745722fe-1525-4d36-b181-28953709d80e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.258089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:43.265677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.266510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.266853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.267194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.268015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:43.269639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.270217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.276535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee5dda-96d4-448c-83fe-d0856cc6bea7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.292190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 371/756] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:43.298757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.356438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.357482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.357983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.358506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.373683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.375557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:43.392361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:43.393428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:43.394416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:43.394416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:43.394416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:43.396540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 386/757] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:43.402506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.425607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdc7ed-76b9-4d46-8b3e-6aeed8f82807 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.431379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:43.469488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.470182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.470463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.470734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.484649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.486791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:43.504379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:43.505387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:43.509494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:43.509494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:43.509494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:43.509935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 372/758] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:43.511438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.561237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5b02-2876-433d-92f2-de738778e757 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.575123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:43.608402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.609301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.609772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.610204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.610993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:43.611434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.613167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb05d0-4b7b-4726-8dbe-7cedd9d29d3b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.630439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 387/759] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:43.632129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e487a-f097-479c-90b6-0a0fb464a713 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.632602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e487a-f097-479c-90b6-0a0fb464a713 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.633047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e487a-f097-479c-90b6-0a0fb464a713 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.634824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 388/760] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:43.639605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e487a-f097-479c-90b6-0a0fb464a713 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.639728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55dcb7-09d8-4f3d-badd-cf41103af4d6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.689368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.690177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.691186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.691472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.692131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:43.692440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.694469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb37ed-4e4e-48c8-b60c-bfcde6fca67b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.715883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 373/761] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:43.715883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.715883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.715883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.734613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:43.736739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:43.738270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:43.771289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e487a-f097-479c-90b6-0a0fb464a713 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.812688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 374/762] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:43.813560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.820348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.821045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.821326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.821696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.829272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.830844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:43.842783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:43.843718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:43.844624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 389/763] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:43.849884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:43.854650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:43.931469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7bdcb-b556-43b8-adb3-d92dd640f026 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.975058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:43.975724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:43.976001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:43.976269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:43.979006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cfa8a-23ae-4e38-bd33-cd4f1d4e94b2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:43.990862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:43.993642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:44.009686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:44.010607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:44.011504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:44.014256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 375/764] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:44.020676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.031199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.031904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.032184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.032454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.033165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:44.033472 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.034960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.036956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:44.050438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 390/765] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:44.056422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.201902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:44.213874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a538f-6a5b-4ec7-8ca7-01c0cf316692 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.217875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81572b9d-d5ae-45f1-bb15-bceecc2fb8b8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.258297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.258924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.259248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.259495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.260196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:44.260497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.260677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.261487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.261979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.262175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.262627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.263731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=585bda89775640bfb817ffc41525181d) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:44.264274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.265996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.272497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 391/766] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/role_assignments?user.id=585bda89775640bfb817ffc41525181d&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:44.273586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.274298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 376/767] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:44.280853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.368045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:44.400612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fb81-febc-4035-b6eb-cb0fd6c2b984 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.417215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141287b-3304-4aa3-bcf2-9297cb2ce5eb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.438879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.439707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.440087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.440721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.443520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=1244993f093f4ad0920e3a42725a2308, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.444526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4893214-f766-4f97-bff6-1e93e13128d6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.462070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 377/768] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:44 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:50:44.463637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6de23e-3110-4c0d-8125-c106d8c628f9 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.463921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6de23e-3110-4c0d-8125-c106d8c628f9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.464196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6de23e-3110-4c0d-8125-c106d8c628f9 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.470322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 378/769] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:44.472225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.472591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.473029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.473237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.473919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.474261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.474530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.475133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.476583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:44.504706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:44.506002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:44.507078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 392/770] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:44.507872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:44.513606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:44.516851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.527339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:44.660817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 379/771] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:44.682773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff29879-7be7-412e-b84e-28234c2bd585 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.706843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:44.719647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.720270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.720549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.720818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.721464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:44.721791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.723179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.730406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 393/772] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:44.735741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.840930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407b421-76a6-4811-878e-0b1fd6c4d410 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:44.892672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:44.893297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:44.893592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:44.893870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:44.894515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:44.894817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:44.896247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:44.901545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 380/773] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:44.910721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.984279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:44.989177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.053229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b23b30-2b78-4445-98e2-6e82ac9f06e3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.096572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.097195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.097467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.097759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.098475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] RBAC: Adding query filter params (interface=public, service_id=cd0b3968ff424c239f244cff3e341bbd, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:45.098781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.100169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.117637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 394/774] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/endpoints?service_id=cd0b3968ff424c239f244cff3e341bbd&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:45.126562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.180694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.185401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccfd5-e9fc-401b-9045-c6ecd6f149cc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.236181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bb0be-fb6e-4368-ba12-0203c95e4fbe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.272857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.273511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.273810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.274082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.274720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.276097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.277712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.278304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.278575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.278839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.279453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.280934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.287115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 395/775] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-32121c7a-a43c-4274-9deb-234790a625a2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:45.293241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:45.296533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:45.297397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:45.300562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 381/776] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:44 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:45.306787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.378303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aeaac-d151-4f78-a4cd-043d922da067 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.378666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aeaac-d151-4f78-a4cd-043d922da067 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.378858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084aeaac-d151-4f78-a4cd-043d922da067 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.380522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 382/777] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:45.394888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.395220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.395441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.426140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:45.427389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:45.444209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3099-0931-495a-a224-13c6170e45c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.459336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.484719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.485339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.485683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.485954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.486676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:45.487006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.488470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.495283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 396/778] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:45.501334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.514344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.520863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 383/779] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:45.616439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061e6d-4978-406c-a0e8-9205cb8972a9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.651655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.653057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.653343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.653653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.654340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] RBAC: Adding query filter params (type=network) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:45.654631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.656151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.665780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 397/780] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:45.668975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.806240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d4c94-ce64-4a41-8769-543a786bcd8e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:45.840877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:45.841914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:45.842419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:45.842897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:45.843831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] RBAC: Adding query filter params (interface=public, service_id=18e4c84ebc83436da009376fc0d2c28e, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:45.844347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:45.845940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:45.854475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 384/781] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/endpoints?service_id=18e4c84ebc83436da009376fc0d2c28e&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:45.861374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.877321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:45.938071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:45.990950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:46.003313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24312454-2f8a-4c64-b705-f8f0ee7ccb95 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.040828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.041776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.042193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.042615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.043376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.045272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.051453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 398/782] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:46.096226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c81f7e-bb20-4134-a2de-2b20482a6a0d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.155316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.155984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.156263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.156536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.177058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:46.182475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:46.183453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:46.184305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 385/783] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:46.189755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.313244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e610b-b010-42b9-bd2f-7207383a9220 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.321180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:46.350109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.351115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.351677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.352147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.353061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:46.353593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.355169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.364507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 399/784] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:46.370127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.504835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c464eb2-0c39-4f60-b4e3-6935f0e37de4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.528196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:46.545646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.546349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.546633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.546902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.559749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.561297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dd4160b1-1068-49d1-8f9f-45917a9b22ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:46.576511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:46.577449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:46.578386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:46.578386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:46.578386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:46.578386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 386/785] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:46.584049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.640739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6c18-f3f6-4952-8872-6d19f547c717 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.640962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6c18-f3f6-4952-8872-6d19f547c717 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.641211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6c18-f3f6-4952-8872-6d19f547c717 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.642658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 387/786] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:46.648503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.648837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.649057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.687240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:46.688459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:46.731827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e449a8b-5894-4b3e-b344-54600983c6c1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.777677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.778676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.779150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.779615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.780483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:46.780979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.782587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.799965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 400/787] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:46.801212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:46.816482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 388/788] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:46.830446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:46.903686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce2c9c-78ca-43f6-98c2-90268f8037a0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:46.938941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:46.940383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:46.940859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:46.941342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:46.949111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:46.950866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a4cea206-d73e-480d-872c-4ac3c9a542ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:46.963118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:46.964121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:46.965070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:46.967580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 401/789] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:46.974721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.055042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ab24-4e10-4304-8df9-91149ce5c492 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.055400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ab24-4e10-4304-8df9-91149ce5c492 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.055564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ab24-4e10-4304-8df9-91149ce5c492 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.057296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 402/790] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:47.064203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.064475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.064744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.085921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:47.087173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.128791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412091f5-057e-4312-a7a0-9f4d064d6752 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.134831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.187627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 403/791] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:47.188495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.189768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.190727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.191237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.191782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.192733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:47.193257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.194972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.207056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.216236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 389/792] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:47.225610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.362862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25879dfa-9eca-4cb0-87e2-e8be93f0004a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.417738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71885098-e0dc-4756-b144-f31d1801686e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.426777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.427685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.428032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.428359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.447390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:47.453333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:47.454280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:47.455109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 404/793] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:47.460447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.500345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.501096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.501381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.501699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.513460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.515013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:47.531205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:47.532216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:47.533116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:47.533116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:47.533116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:47.533116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 390/794] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:47.533841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.554328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.565013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.593242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d09de-7ed2-4b03-96d2-a28d80569893 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.635569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.636182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.636458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.636734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.637413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:47.637758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.639149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.649116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 405/795] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:47.650017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.657693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91d3d0-a541-4c14-b25e-61ace18e4fca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.696856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.697569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.697808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.698080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.709514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7061ad08-a5b4-4b68-8b7d-626cee3d40ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:47.714707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:47.715679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:47.716551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 391/796] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:47.717249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d252a1-7a8a-4b67-ba1f-62116181162b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.717317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d252a1-7a8a-4b67-ba1f-62116181162b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.718225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d252a1-7a8a-4b67-ba1f-62116181162b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.718938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 392/797] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:47.719530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d252a1-7a8a-4b67-ba1f-62116181162b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.765328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4a37-764c-4417-af05-9a98c6df9b05 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.800598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.800781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.801282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.801584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.801852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.802503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:47.802799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.804186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.813584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:47.821620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 406/798] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:47.821620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:47.836180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d252a1-7a8a-4b67-ba1f-62116181162b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.876794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.877438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.877751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.878026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.888861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.890367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-808e3956-fb63-4f6a-a209-59b541129635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:47.904763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:47.905794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:47.906802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:47.906802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:47.906802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:47.906802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 393/799] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:47.907390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.907669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.907945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.926376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:47.927578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:47.951251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fa02-f58b-4807-9adc-8ff778808091 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:47.989039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:47.989959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:47.990369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:47.990770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:47.991563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:47.992095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:47.993705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 394/800] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:47.993896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:47.994561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.014058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 407/801] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.015246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.078619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.081918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.123034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816ce48-e9bc-4e61-8134-031d233ba8db None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.139223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbbf97-01ff-401f-9753-582298f80b23 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.163299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.163917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.164187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.164452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.165297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:48.165638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.167060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.180733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 408/802] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.181636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.187125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.188089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.188611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.189112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.196928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.198396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:48.212217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:48.213112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:48.214056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:48.218183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 395/803] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:48.219165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.309723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa01469-9402-4765-8ea0-e900d4daf968 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.335845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.337933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843cd55b-11b5-422d-a9cf-5f1c54442289 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.348278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.349017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.349295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.349587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.354477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.362775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.364279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.375237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.375873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.376150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.376417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:48.379460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:48.380561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:48.381462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:48.381462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:48.381462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:48.381462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 409/804] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:48.381462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.386813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.388463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:48.400064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:48.401107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 396/805] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:48.402105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.484987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb27e06-f88d-4a4e-ba09-ac1405b3df60 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.500900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6da40b-1246-42bf-86cc-5accdec1ecbb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.518332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.518933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.519204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.519467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.520151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:48.520449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.521908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.533594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 410/806] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.534371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.539355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.540108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.540432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.540781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.541443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.542869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6ff72acb-ff73-4b42-b85a-b5899b2bd869 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:48.552149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:48.553152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:48.554106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 397/807] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:48.555263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d47f3-7a6f-44a5-aaf6-dbecc71a7204 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.555263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d47f3-7a6f-44a5-aaf6-dbecc71a7204 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.555263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d47f3-7a6f-44a5-aaf6-dbecc71a7204 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.556414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 398/808] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:48.557110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d47f3-7a6f-44a5-aaf6-dbecc71a7204 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.579678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.601620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.646509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d518b8-c74c-4c3c-844d-fef80db442c2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.659707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d47f3-7a6f-44a5-aaf6-dbecc71a7204 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.681407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.682046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.682319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.682579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.683213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:48.683503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.684868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.693157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.693792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.694078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.694349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.695079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:48.695402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.695802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 411/809] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.696691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.696956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.716422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 399/810] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.717566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.788783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.796235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e9bed-8e7e-4dfa-a4ac-66f4e2a362e5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.811658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:48.815539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f8e90-b654-47e8-b52e-266ff1896423 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:48.830513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.831164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.831432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.831698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/103c6c2fb3104914b8b4ec7c196f4996/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.833652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=103c6c2fb3104914b8b4ec7c196f4996, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.835084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584e9d8-77d2-4af8-8ade-1356dcda67ce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.848564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 412/811] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:48 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/103c6c2fb3104914b8b4ec7c196f4996/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:48.849734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:48.849859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.850056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.850274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.850370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:48.850618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:48.850891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:48.851530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:48.851834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:48.853269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:48.859322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 400/812] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:48.860212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.881902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:48.883136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:48.963572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 413/813] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:48.965305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:48.988278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be16d55-9d2a-4544-8c3f-b07220047397 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.027703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.028824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.029417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.029962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.037798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.039546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:49.052443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:49.053429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:49.054481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 401/814] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:49.055771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.075396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.086602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.103051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038ae24-0a94-4fe3-af39-4de63696cbe8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.151879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.152532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.152803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.153067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.166756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.168394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.182189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecb270-3984-43e8-828c-db3933afc668 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:49.187678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:49.188661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:49.190376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:49.190376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:49.190376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:49.190376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 414/815] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:49.190376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.228214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.228863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.229465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.229774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.230697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:49.231011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.232638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.239764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 402/816] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:49.240650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.313255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d5817-3713-4798-9d75-84c2271e5f2e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.332499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.349931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03857fc-692c-4d46-ba9e-0dbc29293c48 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.350221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.350800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.351075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.351350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.352086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:49.352393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.353898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.357649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.375573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 415/817] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:49.375573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.390669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.391505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.391617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.391818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.392461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:49.392762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.394180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.489971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c78a81-2d06-49ff-b69e-d935a4419cb7 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.524716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 403/818] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:49.525428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.534912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.535524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.535798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.536064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.536726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.540933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.563141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 416/819] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:49.564024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.626657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.685198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba576bf-8d15-4740-a7bf-2e9927e4425b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.698393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b1092c-41d5-4ad8-a30c-85951df0c7bf None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.738141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.738766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.739043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.739312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.740426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.741072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.741351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.741697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] PATH_INFO: `/v3/services/network` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.742289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.743974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:49.753013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:49.754161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 17 19:50:49.755020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:49.755772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 404/820] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:49.756365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.769185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.830281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.832517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.877506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d6f6e-a90e-4890-a380-dc1ea58e4abe None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:49.891999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:49.917892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:49.918683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:49.919032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:49.919716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:49.927048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:49.928796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:49.941303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:49.942275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:49.943174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:49.947326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 405/821] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:49.948332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:49.971963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 417/822] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:50:49.971963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.039825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.078613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65c85-49c0-4a9f-8409-c783afdeba8d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.115058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11754824-0e98-4cd4-94b0-3b1e5886fc99 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.121345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.122236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.122725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.123246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.124107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] RBAC: Adding query filter params (name=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:50.124616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.126281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.132479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 406/823] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:50.133720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.248352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c3a3e-fd4b-4a85-b8b1-600396677512 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.284742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.285377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.285716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.285991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.288041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.294127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.296302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.296784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.297271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.297866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.298445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:50.298944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.299318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.301089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:50.314286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:50.315267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:50.316172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:50.316172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:50.316172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:50.321301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 407/824] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:50.321477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.321965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 418/825] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:50.323076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.332454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.429988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44e85-2a6e-471b-97be-7492efb3fb58 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.431695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5498087-d5ff-439f-813d-871d8f29c1e0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.466308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.467199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.467621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.468034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.468478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.469087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:50.469435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.469705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.469916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.470280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.471041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:50.471434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.471896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.473022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.482616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 408/826] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:50:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:50.483816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.486749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 419/827] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:50.487728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.541439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.559611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.598967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85677d-a886-4b00-915a-9fb51b22f1e0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.600001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcc34-6188-4a87-9c20-82ec374228ce None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.635321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.635997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.636272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.636580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/585bda89775640bfb817ffc41525181d/roles/c55245851594417e937d6b554d67ca31` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.638527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.639456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.639978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.640454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.641317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.642924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.651790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=585bda89775640bfb817ffc41525181d, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.653366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc7999-f25e-424f-9904-7d1338d4c541 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.668641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 420/828] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:50:50 2022] POST /identity/v3/endpoints => generated 337 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:50.671375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.685449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 409/829] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:50:50 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/585bda89775640bfb817ffc41525181d/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:50:50.811022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3f62f-3c3f-4a4b-8dde-abd9658d8728 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.811409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3f62f-3c3f-4a4b-8dde-abd9658d8728 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.811577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3f62f-3c3f-4a4b-8dde-abd9658d8728 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.813857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 410/830] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:50.819059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.819363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.819543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.837494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3548c4-d28a-4d38-bef9-a7fa3ba03385 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:50.856849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:50.858345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:50.860204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:50.885719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:50.886876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:50.887381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:50.887848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:50.896232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:50.898176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:50.912035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:50.913105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:50.914040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:50.916556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 421/831] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:50.922733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:50.965677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 411/832] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:51.038617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d175f1-462c-435d-9d78-b2875ddf1b20 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.076083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.076704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.076982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.077255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.077992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:51.078292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.079743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.093394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 422/833] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:51.099098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.133099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.207500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b836-ef66-4bad-8caa-64d3f880b8dc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.239382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.242754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.244239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.244525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.244800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.245550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:51.245865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.247292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.255471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 412/834] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:51.287123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.355660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569a0a3-cca0-4053-bf67-5a53f9db5bd5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.395058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.395700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.395971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.396238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.408190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:51.412777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:51.413777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:51.414621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 423/835] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:51.422079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.520853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.527952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30f19-78ca-4aa4-b08c-26c7acf7f303 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.563006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.563617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.563889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.564160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.564820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:51.565115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.566509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86ca35-dd48-464f-ac64-e445c7bfc8d2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.576363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 413/836] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:51.582019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.629802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f4c0-9290-4dd5-8605-deb1b3887049 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.630156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f4c0-9290-4dd5-8605-deb1b3887049 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.630359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f4c0-9290-4dd5-8605-deb1b3887049 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.631996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 414/837] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:51.636971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.637257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.637553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.659565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:51.660783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:51.687611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.693260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b1f3c-02cb-4c22-af49-2dc4245a83a0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.733219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.734587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.735300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.735749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.745086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 415/838] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:51.748835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.750633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:51.765744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:51.766714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:51.767599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:51.767599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:51.767599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:51.769814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 424/839] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:51.778310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:51.867556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:51.883548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453eb45-7cd2-45e2-8c21-93ab44673e27 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:51.918614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:51.919882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:51.920369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:51.920830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:51.921685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:51.922165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:51.923715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:51.935942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 416/840] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:51.944285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.046662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137011b-3d84-4332-9d3a-1a95a8f6bbae None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.062497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.085573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.086556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.087092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.087563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.095498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.097178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:52.108353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:52.109300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 17 19:50:52.110307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:52.112862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 425/841] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:50:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:52.119210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.136558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.181903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8bf24-d727-4bf7-95ea-a2c974091097 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.218944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.219580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.219857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.220129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.220717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.222139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:52.232119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:52.233136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:52.234396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 417/842] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:52.240852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.256390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.265026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9886f4-2f9c-4a97-90f7-df7fddf9b289 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.301315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.302013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.302457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.302807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.303574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:52.303977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.305518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.329249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 426/843] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:52.330417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.374757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f953e-45cc-4447-bf04-d169761e0c64 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.410076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.410883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.411259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.411629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.412367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:52.412808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.414336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.415067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.424982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 418/844] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:52.426772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.460220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ed2b1-1dc7-44c0-a420-1fb248f7ae8c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.496401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.497016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.497288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.497577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.504564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.532340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dfd50-c81f-4874-bdd8-4fd5381a5812 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.570056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.571595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.572076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.572565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.573552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:52.574107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.575689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.580360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.581417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 419/845] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:52.581945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cacf6-852f-46df-b251-b73bb7a09f34 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.657873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.698299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 427/846] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:50:52.704440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.810851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdfa19-bce0-4781-96ae-b1ddd3697e2e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:52.835227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f687-ed48-4a4b-ab4d-d0f0aeff214a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.835406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f687-ed48-4a4b-ab4d-d0f0aeff214a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.835667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f687-ed48-4a4b-ab4d-d0f0aeff214a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.837164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 428/847] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:52.842293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.842570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.842866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.854256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:52.854899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:52.855173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:52.855437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:52.863865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:52.864689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:52.866891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:52.867459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:52.869142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:52.885129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:52.886148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:52.887031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:52.887031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:52.887031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:52.890967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 420/848] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:52.891950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:52.950081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 429/849] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:52.999555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe5c43-b049-4ceb-95c1-5888e0b0c976 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.034689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.035651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.036119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.036601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.037450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:53.037971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.039520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.053024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 421/850] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:53.058337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.156693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.160284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be5102-65f4-4123-8cf6-a4e852351874 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.196707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.197349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.197654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.197924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.204933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.206478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.215537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:53.217762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:53.218603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:53.219488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 430/851] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:53.226480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.279530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.339074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9204-6cc6-4d9b-b718-253c83aef645 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.346201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfded-38ce-4569-9ea9-5a783a550c31 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.373946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.374566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.374843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.375112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.375805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:53.376106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.377639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65882120-3748-49e6-a42a-4928ea1d4b55 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.382893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.383518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.383789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.384139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.389840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 431/852] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:53.391002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80144a-a423-4d80-b5ee-4c317af634e2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.391283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80144a-a423-4d80-b5ee-4c317af634e2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.391548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80144a-a423-4d80-b5ee-4c317af634e2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.392853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 432/853] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:53.395615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80144a-a423-4d80-b5ee-4c317af634e2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.401132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-cbda3192-940f-4da0-9340-d30b9bf10219 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:53.407814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:53.408759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:53.409629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 422/854] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:53.410330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.410330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.410814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.421598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.428773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:53.429995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:53.497607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 423/855] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:53.498658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.541127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.542092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80144a-a423-4d80-b5ee-4c317af634e2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.582932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.583580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.583840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.584089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.584802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=103c6c2fb3104914b8b4ec7c196f4996) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:53.585099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.589339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.601999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 433/856] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/role_assignments?user.id=103c6c2fb3104914b8b4ec7c196f4996&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:53.621714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341983a0-0f59-4b40-99b2-1a5b5f582a21 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.664865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.666846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.667335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.667839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.668785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:53.669314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.671329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.682190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 424/857] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:53.688393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.783137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.796738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.804894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae503a7-6e67-48e6-af41-dc51de176c39 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.841850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.842717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.843127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.843527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.856537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.858252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:53.877570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:53.878851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:53.879894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:53.879894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:53.879894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:53.881919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 434/858] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:53.889582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:53.932565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0b0c-9d7b-4268-bbe4-0ed8440ba545 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:53.965901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:53.981720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:53.982743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:53.983808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:53.984275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:53.985318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:53.987612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:53.995925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a3f7d-5d4c-4dcd-9c8c-421ad15b1e1f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:54.000819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:54.001789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:54.002691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:54.010279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 425/859] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:54.012339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.046939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.047702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.048124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.048487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.049385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:54.050616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.052107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.053696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.068119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 435/860] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:54.074993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.136960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f0098-5a41-4f3b-b7f3-f40c09bac22a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.185714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.186729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.187318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.187485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.188781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:54.189170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ad29d-313a-4da1-945f-3c8fad2b6d8d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.189376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.191416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.201931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 426/861] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:54.203317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.234619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.235588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.236206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.236670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.241735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.247402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.248146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:54.262923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:54.264015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:54.264956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:54.268844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 436/862] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:54.270151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.333530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.337522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6d64c-638a-482f-9291-825ed4169619 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.382881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e39f23-0b95-4c7c-90a1-c18cca33f95a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.385670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.387348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.387767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.388189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.388994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:54.389452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.391024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.398552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 427/863] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:54.434824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.435507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.435840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.436056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.436867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:54.437169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.438739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d0213-bf50-4b8c-b8e2-aabb97adeff4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.465116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 437/864] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:54.467285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.571068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892645-c0ce-4b7f-b693-6d321c9a7e43 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.571310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892645-c0ce-4b7f-b693-6d321c9a7e43 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.571558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892645-c0ce-4b7f-b693-6d321c9a7e43 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.573238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 438/865] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:54.579065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.579269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.579462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.587223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.596446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb5611-1a47-4546-be06-11d3070966a0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.600870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:54.602962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:54.638596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.639223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.639499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.639773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.688629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 439/866] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:54.715250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.716705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:54.815725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.823835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 428/867] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:50:54.830700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.877871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:54.938819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f310c6a-9cac-4824-9fd4-7d3ac5fc576c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:54.970014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:54.975837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:54.976480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:54.976757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:54.977024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:54.988517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:54.989906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:55.005998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:55.007014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:55.007902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:55.007902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:55.007902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:55.012811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 440/868] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:55.013686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:55.084001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca39f9f-1237-4380-9c47-5fb9d064ca0d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:55.120477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:55.121460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.121993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.122161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:55.122592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.123204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:55.124695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.125495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e31c0-2529-462d-8993-8f30a23cc753 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:55.142374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 429/869] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:50:54 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:55.180386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:55.181346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.181821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.182225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.183090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:55.183575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:55.185220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.200659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 441/870] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:55.207203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:55.270558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:55.324242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667c926-5de8-4954-9de2-d4c0a7f0b907 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:55.361256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:55.362602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.363098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.363519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.371245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:55.372813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:50:55.384276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:55.385396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:50:55.386495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:55.388486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 430/871] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:50:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:55.397230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:55.505645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f92c-fa62-4078-8c3d-aaefb2069534 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:55.541851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:55.542754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.543163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.543563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.544732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:55.545260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:55.546842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574b210-3ee4-4c9f-8c19-f83e6fab899e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.559762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 442/872] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:50:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:55.569560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:55.646098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb12f-d3dd-4633-aecf-24304bd3dfc6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.646457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb12f-d3dd-4633-aecf-24304bd3dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.646641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cb12f-d3dd-4633-aecf-24304bd3dfc6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.648342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 443/873] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:55.654093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.654677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.654677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.673967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:55.675167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:55.678626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:55.680106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112a8d-4a58-4342-bc94-859d3535e998 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:55.696380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:55.720984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:55.721672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:55.721953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:55.722210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:55.722935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=585bda89775640bfb817ffc41525181d) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:55.723235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:55.724715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:55.737712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 431/874] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:50:55 2022] GET /identity/v3/role_assignments?user.id=585bda89775640bfb817ffc41525181d&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:55.754448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 444/875] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:56.024524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:56.133240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5753fe7-1cc5-4a34-b57e-7ae3da43a942 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:56.168123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:56.168771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.169049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.169321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.170282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:56.171648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:56.180651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:56.181912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:56.182974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 432/876] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:50:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:56.189990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:56.299364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ffc3a-da11-4450-ade7-f660e7cced5c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:56.334517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:56.335132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.335407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.335682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.336313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:56.336607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:56.338063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05083cb-aa43-48b4-bd86-f14be4c7fed9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:56.343272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 445/877] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:50:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:56.349426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:56.400381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:56.459181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd43fd7-c261-4d43-8ba0-74d58568f565 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:56.491655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3717-f59f-4517-8b4b-34df665d3aec None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.492027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3717-f59f-4517-8b4b-34df665d3aec None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.492359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da3717-f59f-4517-8b4b-34df665d3aec None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.492935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:56.493671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.494020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.494194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 446/878] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:56.494403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.495135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:56.495443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:56.496872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:56.499217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74d30-ada7-44b5-89d7-2cde343b85e5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:56.499566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74d30-ada7-44b5-89d7-2cde343b85e5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:56.499903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74d30-ada7-44b5-89d7-2cde343b85e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:56.502121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 433/879] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:50:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:56.511255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:56.526354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-61b74d30-ada7-44b5-89d7-2cde343b85e5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:56.527693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-61b74d30-ada7-44b5-89d7-2cde343b85e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:56.606440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 447/880] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:56 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:56.874182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:56.980080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de73ac-3d3b-4a3f-a098-ce8c8ab1d12e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:56.987711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:57.015172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:57.016119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.016622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.017101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.018010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:57.019579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c036c-99e4-42b2-ab76-879d2d786c09 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:57.038146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 434/881] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:50:56 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:50:57.206318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54d544-81a6-4362-8a42-c981a297722e None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.206836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54d544-81a6-4362-8a42-c981a297722e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.206836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54d544-81a6-4362-8a42-c981a297722e None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.208265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 448/882] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:57.216741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfbba-f3fd-4678-8d6e-32013d5d1f71 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.217087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfbba-f3fd-4678-8d6e-32013d5d1f71 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.217359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfbba-f3fd-4678-8d6e-32013d5d1f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.236435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f7abfbba-f3fd-4678-8d6e-32013d5d1f71 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:57.237644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f7abfbba-f3fd-4678-8d6e-32013d5d1f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:57.319084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 435/883] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:57.592457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54d544-81a6-4362-8a42-c981a297722e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:57.698854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54d544-81a6-4362-8a42-c981a297722e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:57.734587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:57.735216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.735485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.735747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.736337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:57.737735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:57.737860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca723c1-8dd9-4736-8bdc-004b7556bbf7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.738270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca723c1-8dd9-4736-8bdc-004b7556bbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.738357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca723c1-8dd9-4736-8bdc-004b7556bbf7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.739939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 436/884] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:50:57.740101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:57.746553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.747164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.747732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:57.752193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:57.753167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:57.754423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 449/885] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:57.762458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:57.770204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:57.771437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:57.839560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 437/886] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:50:57.882686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac1bf-2211-423d-961c-55a825ba4ae5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:57.917605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:57.918477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:57.918897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:57.919309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:57.920426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:57.920886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:57.922492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:57.928038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 450/887] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:57.935130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:57.959908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.035213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420203df-bbac-47ad-86a6-9090c3f9df8b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.073283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.074302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.074780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.075250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.076090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:58.076611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.078206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.083996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 438/888] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:50:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:58.090737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.103919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.155775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.200905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0d7e8-31b5-4432-8256-7374563c4aae None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.212007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.230307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540df92c-6ab2-4975-abf2-f7fce82404f5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.249927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.250623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.250819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.251075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.252197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] RBAC: Adding query filter params (interface=public, service_id=ce8016aa57dd4b4facae4747a4cc8c95, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:58.252503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.253918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.261570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 451/889] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/endpoints?service_id=ce8016aa57dd4b4facae4747a4cc8c95&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:58.267704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.274605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.275372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.275708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.276048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.287641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:50:58.292754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:58.293771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:58.294977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 439/890] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:58.301582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.376168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5983f-b52c-48d7-85cf-626568a056cd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.401987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.410767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.411655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57cd0-391c-49ad-ace5-716b9cb8a78f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.411854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.412297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.412860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.413825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.415606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d8d84-b22f-40df-86ea-f63513b022a2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.422359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 452/891] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:50:58.447803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd24fb-e7d0-4484-96b9-aed3c273af77 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.448207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd24fb-e7d0-4484-96b9-aed3c273af77 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.448475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd24fb-e7d0-4484-96b9-aed3c273af77 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.450309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 453/892] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:58.453675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.454497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.455051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.455467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.456284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:58.456731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.458449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.459511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.459886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.460254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.477230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 440/893] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:58.478525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.483250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:58.484668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:58.493065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.569256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 454/894] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:58.611870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb8c79-f5fc-42cb-a10c-eac610cea730 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.646899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.647805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.648217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.648638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.660524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.662299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:58.677326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:58.678433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:58.679446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:58.679446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:58.679446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:58.681243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 441/895] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:58.686837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.791387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d26b2-a240-41af-a726-2f5d541167d6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.812824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.827112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:58.827721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:58.827995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:58.828261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:58.828912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:58.829208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:58.830620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:58.839737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.842696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 455/896] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:58.848138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:58.917618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:58.967666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7eec8-c8e3-45fa-a140-d46cfa712572 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:58.976475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb64a6-e6c5-4fe8-910f-bf30ae43476f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.009754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.011109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.011830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.012281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.019162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.019853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.020243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.020620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.021673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.022433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.023184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dacf6-66ad-49d1-8647-bcba8eb9d40d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.024245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:50:59.037989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:59.039095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:50:59.040090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:59.042642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 456/898] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:50:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:59.045601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 442/898] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:50:58 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:50:59.048406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.106703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.116325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d189a-3003-4c10-a2d1-7aeabbfcf41f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.116687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d189a-3003-4c10-a2d1-7aeabbfcf41f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.116887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d189a-3003-4c10-a2d1-7aeabbfcf41f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.118619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 443/899] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:50:59.124265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.124922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.125419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.149319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:50:59.150706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:50:59.183091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836fda6-da95-4816-8788-5436cb720140 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.231113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.232054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.232054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.232321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.233044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:59.233343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.234936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.245571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 444/900] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:50:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:50:59.255079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 457/901] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:50:59.260983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.312794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.363403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10ec60-faaa-4b3e-abfd-b677159dc1ac None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.401240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.401940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.402221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.402496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.472153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.473530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.522913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.532284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.591933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 445/902] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:50:59.597184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.654633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.663433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689f872-68c4-4fc0-81ca-09360f6968d8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.707390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.708293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.708568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.708843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.709756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.711895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.720170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366503c-0fa5-49c2-a095-4b41c1aafbaf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:59.723616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:59.724567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:50:59.725501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 458/903] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:50:59.731028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.760520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.761161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.761435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.761739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.772955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.774504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:50:59.790780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:50:59.792966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:50:59.794297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:50:59.794297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:50:59.794297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:50:59.794297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 446/904] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:50:59.800404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.806069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.845762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ecb2c-dd49-46e6-8f06-01f0f6fe25f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.882692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.883637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.884166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.884639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.885645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:59.886195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.887840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:50:59.893500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 459/905] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:50:59.898919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:50:59.924932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:50:59.934517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ddc715-3111-45a7-a873-ac22af86676a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:50:59.984354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:50:59.985405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:50:59.986314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:50:59.986742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:50:59.987795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:50:59.988288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:50:59.990470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.011014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 447/906] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:00.011228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e767004-00cb-4f2f-b6e8-f7a5c8488173 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.012668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.056535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.057905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.058941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.059428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.060083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:00.060381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.061829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.072329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 460/907] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:50:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:00.073747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.082032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:00.132779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549f982-e90a-47c7-ad55-6ebc96553093 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.168204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.168846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.169124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.169395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.177345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.178755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.185055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:00.190176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:00.192642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:00.193765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 448/908] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:00.198917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.207105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab539f9b-0506-439c-9d8d-54fdfdd51954 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.244469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.245427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.245869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.246298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.247188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] RBAC: Adding query filter params (interface=public, service_id=2066f3efbf8b40f5a91322f3e1410680, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:00.247650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.249279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.255694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 461/909] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/endpoints?service_id=2066f3efbf8b40f5a91322f3e1410680&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:00.265598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.320824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6115-44da-4f68-b972-89986970a9e4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.344270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:00.357743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.358363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.358689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.358965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.359663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:00.359967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.361399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.374549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 449/910] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:00.380412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.384417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6690-9026-47d7-9435-2beafb84ca78 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.423615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.424220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.424492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.424757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.425367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.426857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb419ac-10d9-4f2d-b0d9-54e852ba2d15 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.432833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 462/911] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:51:00.434099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab1a7-8950-4a79-8db4-c9b6307cddc5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.434378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab1a7-8950-4a79-8db4-c9b6307cddc5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.434646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dab1a7-8950-4a79-8db4-c9b6307cddc5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.437693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 463/912] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:00.441659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.441982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.442251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.461975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:00.463190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:00.473890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:00.497787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f985d-fffd-4ea2-9ccb-7ed379a9d925 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.535959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 464/913] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:00.555141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.556264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.556803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.557318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.558355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] RBAC: Adding query filter params (role.id=4ff5a2dfdb484e0ea7887574d77b3025, scope.project.id=327ecf4949a040998a1557337864548d, user.id=585bda89775640bfb817ffc41525181d) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:00.558861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.560866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.568283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 450/914] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/role_assignments?user.id=585bda89775640bfb817ffc41525181d&scope.project.id=327ecf4949a040998a1557337864548d&role.id=4ff5a2dfdb484e0ea7887574d77b3025 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:00.813001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:00.915373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677fb4-137c-4565-8f2c-75861c9e4a8e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:00.952089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:00.953142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:00.953651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:00.954128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:00.954917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:00.956518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2884dbfb-3cba-4bd1-9ef7-44275348ea74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:00.965741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:00.966749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 17 19:51:00.967608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:00.970346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 465/915] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:00.976561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.026193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:01.084487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca17b-0bf2-450b-bfef-fb8d9009550f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.086522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b39192-ab6c-4c7a-95ed-71098f81faee None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.087071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b39192-ab6c-4c7a-95ed-71098f81faee None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.087517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b39192-ab6c-4c7a-95ed-71098f81faee None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.089337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 466/916] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:51:01.098291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.098569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.098840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.118338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:01.119645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:01.121857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:01.125880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.126810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.127284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.128171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.129011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:01.129500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.131764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.139516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 451/917] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:51:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:01.145268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.201659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 467/918] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:01.243233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7656c-f3d7-4f84-a9fd-175017bc7ce5 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.253872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:01.279683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.280526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.280796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.281065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.281745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:01.282046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.283408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.288812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 452/919] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 17 19:51:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:01.294839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.399484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc026de-e220-4983-80ce-6ff6e0cf1486 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.434602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.435214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.435489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.435756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.436430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.437839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.446436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:01.462886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 468/920] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:01.492093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.605647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b247-ea5b-48ab-a989-3879562ea0b2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.647489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.648123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.648396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.648658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.649231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.650716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:01.660021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:01.661072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:01.662035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 453/921] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:01.668255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.779057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcdd434-292f-429d-a4df-c294b67399b6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.813306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.813943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.814215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.814478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.815187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:01.815487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.817065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.822699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 469/922] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:01.828607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:01.867679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:01.936125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9799d-97fb-4d24-a666-75e07e82cc10 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:01.970431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:01.971049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:01.971317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:01.971585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:01.972225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:01.972522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:01.973906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:01.979625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 454/923] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:01.986282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:02.050402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:02.112756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dbac9-98cf-4316-b19a-098d9c82fa98 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:02.153086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:02.154141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.154630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.155097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.156003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] RBAC: Adding query filter params (interface=public, service_id=60858dee15ec444089ad17a06e0c6203, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:02.156507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:02.158174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884523c1-da0e-41aa-8682-05ad9e96807b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:02.164380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 470/924] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:51:01 2022] GET /identity/v3/endpoints?service_id=60858dee15ec444089ad17a06e0c6203&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:02.170254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:02.275446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584d414-b262-4450-aeb1-f7116f7c42b3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:02.276220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:02.309886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:02.310496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.310768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.311036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.311639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:02.312990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e51fd-65f7-46a9-9472-0a07bb4baec6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:02.319361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 455/925] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:51:02 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:51:02.383097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00ba40-6e87-4887-a26f-d3006757ec66 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.384179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00ba40-6e87-4887-a26f-d3006757ec66 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.384673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00ba40-6e87-4887-a26f-d3006757ec66 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.386630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 471/926] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:02.392660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a67-1f5f-4103-a5a2-3e6f19088c8e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.393307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a67-1f5f-4103-a5a2-3e6f19088c8e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.393821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2a67-1f5f-4103-a5a2-3e6f19088c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.413367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c6c2a67-1f5f-4103-a5a2-3e6f19088c8e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:02.415167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c6c2a67-1f5f-4103-a5a2-3e6f19088c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:02.492126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 456/927] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:02.595344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42c7a-a064-4f4a-b1d1-ef94d6d69088 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.595982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42c7a-a064-4f4a-b1d1-ef94d6d69088 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.596466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42c7a-a064-4f4a-b1d1-ef94d6d69088 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.598308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 472/928] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:02.603860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.604144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.604414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.623237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:02.624739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:02.695206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 457/929] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:02.765046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc42c7a-a064-4f4a-b1d1-ef94d6d69088 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:02.869928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc42c7a-a064-4f4a-b1d1-ef94d6d69088 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:02.892997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:02.906104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:02.906746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:02.907020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:02.907291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:02.907884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:02.909262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:02.918265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:02.919171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:02.920487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 473/930] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 17 19:51:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:02.925976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:02.951171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:02.959217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.046214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815eabe6-b748-4c82-9b25-02207c2e2c63 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.078061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d50e50-64f1-4888-92c0-c3cbaab0ee7d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.091468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.092336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.092625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.092899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.093574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:03.093896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.096030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.105685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 458/931] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:03.108070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.121278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.122154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.122570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.122977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.135050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:03.140182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:03.141332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:03.142217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:03.149668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 474/932] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:51:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:03.151390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.203428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.222210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.242755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f03b53-5433-4162-b452-3f40df38022e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.270454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c2e9d-bbe7-4eb7-a6b8-970a2b79ac63 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.284254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.285249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.285852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.286360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.287616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:03.288532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.291828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.300248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 459/933] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:03.305629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.318103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.319114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.319406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.319679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.320357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:03.320832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.322683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.338429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 475/934] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:03.339630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.424613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d154df2-a95f-4daf-bf73-82ec56061cb0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.468465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.469079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.469354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.469649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.470312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.471671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e360ef-a1ab-48e6-a41f-d49df65d6498 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.479940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614672e-0aa2-4316-937d-a60ab98a20b0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.494953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.504043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 460/935] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:03.540257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.541186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.541689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.542104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.555862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.557748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.563734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb2c-8fce-4e90-8b46-c72db0abcb88 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.564315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb2c-8fce-4e90-8b46-c72db0abcb88 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.564722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb2c-8fce-4e90-8b46-c72db0abcb88 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.566314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 461/936] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:51:03.577051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.577361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.577602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:03.582150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:03.583139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:03.584052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:03.584052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:03.584052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:03.586383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 476/937] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:03.594102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.594302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.600819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:03.602242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:03.698287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 462/938] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:03.717035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cef41-930f-4944-a114-4c64a9f7e0df None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.756955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.757594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.757879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.758146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.758832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:03.759135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.760581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.773428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 477/939] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:03.779139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.883779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7674-71cb-40aa-b815-0ca41eca6165 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:03.920144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:03.920789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:03.921066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:03.921335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:03.924270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.929568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:03.931053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:03.943275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:03.944428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:51:03.945634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:03.947421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 463/940] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:03.954731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:03.974433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:03.997070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.067925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f2dd1-12f7-4477-b28a-2630a5c1985f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.101494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965e466-4adb-44a5-8960-b493cd84eea4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.108101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.109459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.109844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.110184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.111032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:04.111832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.113949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.136094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 478/941] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:51:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:04.141637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.146757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.147379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.147657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.147966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.148636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.150182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.153011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING keystone.server.flask.common [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 17 19:51:04.177734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.sql.core [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:51:04.177734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 19:51:04.177734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [parameters: {'id': '0f74bd54cb0a40dcbc7f3ff8888cc18a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 19:51:04.177734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0f74bd54cb0a40dcbc7f3ff8888cc18a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0f74bd54cb0a40dcbc7f3ff8888cc18a. Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:51:04.193748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 17 19:51:04.194630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application yield resource Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise self.value Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application session.commit() Aug 17 19:51:04.195812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:51:04.197108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:51:04.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:51:04.199356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [parameters: {'id': '0f74bd54cb0a40dcbc7f3ff8888cc18a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:04.201685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0f74bd54cb0a40dcbc7f3ff8888cc18a. Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.202859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 464/942] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 17 19:51:03 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:04.205414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.250727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.262153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.287335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea620e-5e63-4043-9c62-29adb41b9dfd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.317338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985700c-0ccd-452a-8906-9f853844a5ef None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.336011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.337058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.337951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.338229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.349330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.350908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.352653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.353448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.353604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.353878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.360596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.362226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b875e32d-96ac-4c76-8115-692cd84dd299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:04.365708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:04.366630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:04.367526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:04.367526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:04.367526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:04.367526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 479/943] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:04.368569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5471a8-838b-4771-822d-4fc0a270c998 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.368844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5471a8-838b-4771-822d-4fc0a270c998 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.369109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5471a8-838b-4771-822d-4fc0a270c998 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.370475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 480/944] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:04.372659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5471a8-838b-4771-822d-4fc0a270c998 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1d102fb4-8870-49f0-9ff3-398dba724579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:04.379001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:04.380051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:04.381095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 465/945] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:04.381724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.382005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.382275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.400324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:04.401784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:04.471323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 466/946] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:04.472511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.488770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5471a8-838b-4771-822d-4fc0a270c998 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.519672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.522449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.533017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.533666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.533940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.534209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.535129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:04.535445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.536979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.555999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 481/947] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:04.564912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.603441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fbeb1-f9b5-4606-b6c2-469be42ebe52 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.646417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.647039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.647675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.647949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.648895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:04.649197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.651149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.667315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4566932-6a24-4fdf-b47b-68dff3ef949f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.680110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 467/948] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:04.716116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.716936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.717268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.717629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.725782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.727438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:04.740687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:04.741706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:04.742596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 482/949] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:04.748615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.754417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.829973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.887922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9af620-52b9-4f65-908f-1fe6d37a4483 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.888708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcec37f-7790-4c50-94b9-fa249a9aa987 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:04.893844 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:04.932097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.932724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.932996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.933263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.934277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:04.934277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.935697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.937759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:04.938775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:04.939052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:04.939320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:04.939914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:04.941277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:04.950872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 468/950] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:04.960234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:04.961465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:04.962694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 483/951] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:04.963663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:04.963752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:05.062993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaebeeb-76e9-4200-a0d9-84fe76f4a7ce None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:05.070291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d002749-9096-4140-9b3c-57d4b341bc3a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:05.081964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:05.097819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:05.098653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.099027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.099396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/585bda89775640bfb817ffc41525181d/roles/4ff5a2dfdb484e0ea7887574d77b3025` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.101896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=585bda89775640bfb817ffc41525181d, role_id=4ff5a2dfdb484e0ea7887574d77b3025)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:05.103461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:05.108408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:05.109151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.110034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.110881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.111896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:05.112301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:05.113956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:05.125594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 484/952] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:05.125795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 469/953] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:51:04 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/585bda89775640bfb817ffc41525181d/roles/4ff5a2dfdb484e0ea7887574d77b3025 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:51:05.142045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:05.260008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af40ec-bc74-4992-9c8d-20c403ddee97 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:05.297441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:05.298303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.298670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.299034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.299766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:05.300153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:05.301723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fea293-a242-4941-9ad2-8425b563f395 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:05.307501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 485/954] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:05.313946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:05.349593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:05.436181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9dced-360c-42c0-9bda-6ba2a0f8982e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:05.478344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:05.479137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.479507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.479876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.480648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:05.482148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73623b-cb73-4bd4-9fd9-1afbb456ad9e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:05.505673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 470/955] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:05.570262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643ca-2943-4767-bc2e-f53b5e509181 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.570786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643ca-2943-4767-bc2e-f53b5e509181 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.570786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643ca-2943-4767-bc2e-f53b5e509181 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.574317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 486/956] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:05.580278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:05.580572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:05.580859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:05.600790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:05.602070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:05.684798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 471/957] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:05.954569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5643ca-2943-4767-bc2e-f53b5e509181 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:05.997003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:06.060328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5643ca-2943-4767-bc2e-f53b5e509181 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:06.094570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:06.095201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:06.095473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:06.095740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:06.096323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:06.097721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:06.106478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:06.107553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:06.108450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 487/958] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:06.113820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:06.122083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:06.218136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a51dc-79f1-4258-9c57-c0aab296fa1e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:06.252723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11730099-6556-4297-a775-09c8d8978100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:06.253340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730099-6556-4297-a775-09c8d8978100 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:06.253647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730099-6556-4297-a775-09c8d8978100 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:06.253925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11730099-6556-4297-a775-09c8d8978100 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:06.254565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11730099-6556-4297-a775-09c8d8978100 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:06.254871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11730099-6556-4297-a775-09c8d8978100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:06.256236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11730099-6556-4297-a775-09c8d8978100 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:06.261524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 472/959] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:06.266910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:06.372453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03df06-770b-49d2-bc81-aff0a653fa27 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:06.374031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:06.407981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:06.408620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:06.408889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:06.409154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:06.409828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:06.410125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:06.411557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a247c-2e90-4af2-a755-9dafc4475957 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:06.416728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 488/960] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:06.732835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894acc1-ed43-461f-8a8c-1df341b25a2c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:06.733212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894acc1-ed43-461f-8a8c-1df341b25a2c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:06.733388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894acc1-ed43-461f-8a8c-1df341b25a2c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:06.735152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 473/961] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:06.740491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea664c11-0913-4e18-a547-757fdf5cbf40 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:06.740934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea664c11-0913-4e18-a547-757fdf5cbf40 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:06.741030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea664c11-0913-4e18-a547-757fdf5cbf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:06.759461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ea664c11-0913-4e18-a547-757fdf5cbf40 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:06.760664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ea664c11-0913-4e18-a547-757fdf5cbf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:06.835524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 489/962] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:07.104637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894acc1-ed43-461f-8a8c-1df341b25a2c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.109117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2899-ac50-41f4-8195-be1c8f47140c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.109401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2899-ac50-41f4-8195-be1c8f47140c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.109713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2899-ac50-41f4-8195-be1c8f47140c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.111332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 490/963] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:51:07.116227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.116505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.116776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.136738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:07.138601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:07.176909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:07.217145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894acc1-ed43-461f-8a8c-1df341b25a2c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.218471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 491/964] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:07.253479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:07.254122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.254397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.254666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.255305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:07.256724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:07.273923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.sql.core [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:51:07.273923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 19:51:07.273923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [parameters: {'user_id': '913ee65f56c643e9a93c75c3242dfba8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 19:51:07.273923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81532) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 17 19:51:07.281133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:51:07.282039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application yield resource Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application next(self.gen) Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.rollback() Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.force_reraise() Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise self.value Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application session.commit() Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 17 19:51:07.283006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._prepare_impl() Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.session.flush() Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._flush(objects) Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application compat.raise_( Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application flush_context.execute() Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rec.execute(self) Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 17 19:51:07.283905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 17 19:51:07.284798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result = self._query(query) Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application conn.query(q) Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application result.read() Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 17 19:51:07.288372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [parameters: {'user_id': '913ee65f56c643e9a93c75c3242dfba8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:07.290412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 17 19:51:07.291626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:07.292786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 474/965] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 17 19:51:07 2022] POST /identity/v3/users => generated 160 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:07.297616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.334155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:07.402799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2798c-8983-4f9b-97ad-6bf8a1cd6205 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.437367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:07.438376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.438865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.439338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.447055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:07.448765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:07.460517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:07.461496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 17 19:51:07.462511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:07.464980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 492/966] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:07.470971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.484139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.593324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc4b90-7fe0-44b1-9b77-cf1a97ad163e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.594552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:07.611542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e62bf7-8a54-434d-b87a-1357b6fb088a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.622163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:07.629330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:07.630373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.630840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.631109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.631975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:07.632281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:07.633981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535cc8b-2315-453a-b600-701dc3e38978 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:07.648385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:07.649023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.649328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.649635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.660254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-abab65fe-edc9-498b-b2e6-ab5d9eb3b28d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:07.665378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:07.666449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:07.667308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 493/967] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:51:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:07.668019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2947a-71c5-48b4-8990-7def07ccd88f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.668019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2947a-71c5-48b4-8990-7def07ccd88f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.669109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2947a-71c5-48b4-8990-7def07ccd88f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.673587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 494/968] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:07.673587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2947a-71c5-48b4-8990-7def07ccd88f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.689564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 475/969] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 17 19:51:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:07.689564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.689564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.689564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.707584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:07.709420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:07.801182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2947a-71c5-48b4-8990-7def07ccd88f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.808174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 476/970] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:07.844969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:07.845636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:07.845913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:07.846181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:07.846849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:07.847146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:07.848514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:07.858432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 495/971] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:07.863823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:07.972219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6384a-5af8-489a-aeff-927bb33ab940 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:07.990949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.008317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.008983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.009262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.009546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.019875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.021316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:08.035939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:08.036938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:08.037889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:08.037889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:08.037889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:08.041610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 477/972] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:08.043313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.054993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.074986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.151314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a430e-7c6d-45a7-8900-4d04379e8444 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.178894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16555b-b1c5-49be-8b66-b24675ddda58 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.187790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.188401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.188673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.189051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.189730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:08.190047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.191432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.203756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 496/973] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:08.209234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.212800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.213517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.213826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.214125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.214714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.216080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:08.224923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:08.226001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:08.227239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 478/974] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:08.231869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.267151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.282355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.320007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46284c-2c23-426f-ab06-0a9211c38208 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.340081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44c8bd-1453-4f87-b121-c04c7397a77c None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.356554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.357141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.357413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.357932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.365387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.366866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.375611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.376226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.376506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.376780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.377413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:08.377746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2f54a031-51a8-41b8-bd82-5a8aa8ba4cca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:08.377904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:08.378802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:08.379743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 497/975] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:08.380365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5b4e-cbdb-42b4-89f7-f3eae2fe1685 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.380365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5b4e-cbdb-42b4-89f7-f3eae2fe1685 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.380781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d5b4e-cbdb-42b4-89f7-f3eae2fe1685 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.381847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 498/976] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:08.384124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4507e8-8b16-4713-b6ec-6394b8a4f65e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.384965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d5b4e-cbdb-42b4-89f7-f3eae2fe1685 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.389522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 479/977] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:08.390747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.391034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.391308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.409469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:08.410727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:08.476938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 480/978] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:08.477832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.491056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d5b4e-cbdb-42b4-89f7-f3eae2fe1685 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.507810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.521640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.531865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.532493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.532767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.533049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.533824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:08.534130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.535579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.555311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 499/979] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:08.561272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.594438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4c71a-caf2-4a51-ae61-bb9ac9105650 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.629509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.630152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.630435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.630708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.631376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:08.631680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.633046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.637918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 481/980] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:08.643790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.669122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf63299-3ff7-4f3f-95dd-4708e3dcd0ec None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.710482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.711088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.711358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.711622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.736638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.746386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d538df0-ca3d-4662-90a2-24c749ea2233 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.770244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.779511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.780121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.780398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.780666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.781381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] RBAC: Adding query filter params (interface=admin, service_id=60858dee15ec444089ad17a06e0c6203, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:08.781709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.783069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.784137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.785616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.788171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 482/981] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/endpoints?service_id=60858dee15ec444089ad17a06e0c6203&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:08.788927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.883659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 500/982] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 19:51:08.884508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.887101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74ec58-4ff8-4f2a-9ceb-b8cbe92f22b1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:08.921035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:08.921667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:08.921947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:08.922238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:08.922885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:08.924284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:08.939339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 483/983] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:51:08 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:08.940134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:08.959726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:08.985177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:09.008047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d519b99b-4ffe-41cc-9697-4559f53ab2af None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.054193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.054824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.055097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.055361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.056144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.057616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.063200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bc0d9-8be5-4428-89cf-67cc568931cc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.065083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 501/984] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:51:09.105668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.106318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.106590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.106851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.119358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.120835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:09.136507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:09.137835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:09.138941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:09.138941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:09.138941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:09.138941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 484/985] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:09.144151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:09.253817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8b17d-c5bb-4c0f-8d23-03852b8c8679 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.288431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.288960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.289235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.289499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.290240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:09.290536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.291943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.304527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 502/986] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:09.310806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:09.377779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:09.416230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f802-e0d1-45b0-822d-0f7173a7e666 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.451941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.452576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.452813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.453112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.460097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.461521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:09.472801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:09.473809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:09.475095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 485/987] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:09.479804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:09.493169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:09.596136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bff58-90a4-48f6-961b-44cc5ad43d77 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.632127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.633107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.633607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.634087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.634963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:09.635476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.637109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f47e-6fa3-41dc-95b7-bd46d93e2361 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.650349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 503/988] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:09.656496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:09.757161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68289a-df55-46cf-bc14-a0be8e6f527f None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:09.782903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:09.792417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:09.793036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:09.793557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:09.793846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:09.794895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] RBAC: Adding query filter params (role.id=4ff5a2dfdb484e0ea7887574d77b3025, scope.project.id=327ecf4949a040998a1557337864548d, user.id=585bda89775640bfb817ffc41525181d) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:09.795217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:09.796639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5f38-395e-4727-a8c6-bea9d89d5d45 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:09.813579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 486/989] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:09 2022] GET /identity/v3/role_assignments?user.id=585bda89775640bfb817ffc41525181d&scope.project.id=327ecf4949a040998a1557337864548d&role.id=4ff5a2dfdb484e0ea7887574d77b3025 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:10.876726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac93d-83b6-4659-abf7-f27bab5d842a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:10.877275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac93d-83b6-4659-abf7-f27bab5d842a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:10.877275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ac93d-83b6-4659-abf7-f27bab5d842a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:10.878816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 504/990] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:10.883925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:10.884353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:10.884449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:10.903883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:10.905116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:10.954133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4954b-b097-4127-9464-f8cba44808fd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:10.954474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4954b-b097-4127-9464-f8cba44808fd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:10.954638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4954b-b097-4127-9464-f8cba44808fd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:10.956209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 505/991] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:51:10.960816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:10.961089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:10.961353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:10.976591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 487/992] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:10.979357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:10.980551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:11.042933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.052830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 506/993] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:11.238029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.311018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.345850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971a967-bb05-4c9e-8418-0c770dd91cc7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.369700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.382222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.383034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.383214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.383474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.384137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.385604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:11.397453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:11.398512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:11.399367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 488/994] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:11.407871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.423956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6b3e4-bc03-409c-9b39-4af93925ebfe None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.461480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.462146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.462421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.462691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.463272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.464656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:11.473327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:11.474353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:11.475206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 507/995] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:11.480331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.517819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533afc7-176b-4407-a00d-99a29cb87bdf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.544330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.568534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.568534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.568534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.568534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.568957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:11.568957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.569567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.579826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 489/996] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:11.581482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.598138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfaa83-0b9d-4bc7-b4f8-d63f3952195b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.634740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.639647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.640263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.640533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.640794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.641429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:11.641754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.643138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.648990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 508/997] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:11.654918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.691630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ecf7d-5714-46c0-a928-1aa7b2f6e9ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.727513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.728132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.728397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.728665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.729321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:11.729642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.730976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642386a-2a84-4a75-89d7-212c047bfae7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.736094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 490/998] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:11.737250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfbfac-6a87-4a3e-8d21-bef8351417df None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.737527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfbfac-6a87-4a3e-8d21-bef8351417df None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.737821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfbfac-6a87-4a3e-8d21-bef8351417df None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.741631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 491/999] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:11.743345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfbfac-6a87-4a3e-8d21-bef8351417df None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.760631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53590a4b-2de7-4050-bc46-0cd6997fee02 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.784010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.794565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.795510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.795781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.796036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.796654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:11.796943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.798340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559369c8-366c-4c91-82d8-8e283de87d10 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.803218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 509/1000] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:11.804293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.804570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.804835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.821952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:11.823143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:11.848622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfbfac-6a87-4a3e-8d21-bef8351417df None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:11.862008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:11.882167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:11.882779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:11.883057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:11.883327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:11.884027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] RBAC: Adding query filter params (interface=public, service_id=c853b63209054f0099903cace094fb4a, region_id=RegionOne) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:11.884578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:11.885709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:11.886983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 510/1001] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:11.887801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.898634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 492/1002] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/endpoints?service_id=c853b63209054f0099903cace094fb4a&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:11.904400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:11.991946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f3b3a-6076-4192-ba33-e2ebec17356c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.009040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c293fa-d65e-4b95-8d99-6bd73e6fe200 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.009905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:12.028072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.028791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.029061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.029324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.030085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.031773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.042746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.043606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.044058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.044555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.045824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.047488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.053585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 511/1003] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:12.064406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 493/1004] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:51:11 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:51:12.155613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:12.277440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4927b-dcf3-48fc-b399-b9fe8ea3767a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.313741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.314306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.314582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.314848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.315440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.316834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:12.326264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:12.327287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:12.328127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 512/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 17 19:51:12 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:12.333315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:12.379104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:12.439782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb11c0-776d-4fd6-95d7-1254c0ee494d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.475646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.476260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.476538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.476811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.477486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:12.477848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.479217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a00db0-4cea-44ec-bc36-f04cf0354dd5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.484435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 494/1006] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:12.490276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:12.531968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:12.624057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6f05f-a176-45b9-8748-8711af971fcb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:12.660399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:12.661034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:12.661313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:12.661612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:12.662266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:12.662564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:12.663946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f903b-15be-4859-800b-b91fb3da40bd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:12.673671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 513/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:13.984181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab531ee-6c82-420c-8831-c71d09a0861d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:13.984768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab531ee-6c82-420c-8831-c71d09a0861d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:13.984768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab531ee-6c82-420c-8831-c71d09a0861d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:13.989572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 495/1008] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:13.991616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447e188-3267-4671-9d94-262f6209c82f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:13.992085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447e188-3267-4671-9d94-262f6209c82f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:13.992346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447e188-3267-4671-9d94-262f6209c82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.011257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4447e188-3267-4671-9d94-262f6209c82f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:14.012458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4447e188-3267-4671-9d94-262f6209c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:14.024704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e6cb-0586-4131-acb6-d7c3b6ed12cb None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.024995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e6cb-0586-4131-acb6-d7c3b6ed12cb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.025268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e6cb-0586-4131-acb6-d7c3b6ed12cb None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.026851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 496/1009] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:14.031599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.031885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.032166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.050838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:14.052040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:14.090346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 514/1010] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:14.123340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 497/1011] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:14.355215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447e188-3267-4671-9d94-262f6209c82f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.385325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.388175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:14.424798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:14.465124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447e188-3267-4671-9d94-262f6209c82f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.496280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953a3eb-d591-4a81-9a60-7668ca05e151 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.500760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.501404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.501727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.501995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.503118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.504576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.506424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 515/1012] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:14.511969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.533505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.534169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.534448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.534720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.535308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.536674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4d307ca0-e445-4680-9e2c-8f49a31aa6d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:14.546001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:14.547187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 17 19:51:14.548315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:14.549285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 498/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:14.550784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718946-4585-435b-a751-f02fb1d6bb69 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.551068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718946-4585-435b-a751-f02fb1d6bb69 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.551335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e718946-4585-435b-a751-f02fb1d6bb69 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.552928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 499/1014] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:14.553684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e718946-4585-435b-a751-f02fb1d6bb69 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.610285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:14.612435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7573171-9121-4271-b5de-b27e438f1b4b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.646528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.647143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.647447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.647713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.648291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:14.648390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.649905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192db9-91fd-4a14-bc68-bf8e36969cf6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.652497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e718946-4585-435b-a751-f02fb1d6bb69 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.677402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 516/1015] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:51:14 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:51:14.678815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.679101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.679368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.693040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.693804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.694005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.694313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.694921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:14.695227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.696700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.707409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 500/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:14.708064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:14.708728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.709317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:14.785148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 517/1017] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:14.817777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777999f-bfab-4b61-b948-22df132afe83 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:14.853237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:14.853860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:14.854153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:14.854410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:14.855060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:14.855384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:14.856726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:14.861738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 501/1018] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 17 19:51:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:14.867697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:14.951217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:14.972909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8565e-d195-4d26-9c30-70dc03f3363f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:15.008356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:15.009064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:15.009283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:15.009550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:15.010220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:15.011610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0650eb-4186-43eb-a850-1105b462d742 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:15.038820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 518/1019] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:15.050867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:15.081866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:15.165970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99202f5b-8992-474a-ad30-3e41fd6289ed None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:15.214752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:15.215481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:15.215756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:15.216027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:15.216690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:15.218072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b0508-6c3e-4d10-b410-65b4f7f8a275 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:15.236283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 502/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:51:15 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:51:16.629854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c075-296f-46a4-a508-68645825652a None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:16.630414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c075-296f-46a4-a508-68645825652a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:16.630414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c075-296f-46a4-a508-68645825652a None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:16.631852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 519/1021] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:16.637370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1bfba-10ca-4a03-ac8d-ca63cffe32ce None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:16.638012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1bfba-10ca-4a03-ac8d-ca63cffe32ce None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:16.638012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1bfba-10ca-4a03-ac8d-ca63cffe32ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:16.657036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-26a1bfba-10ca-4a03-ac8d-ca63cffe32ce None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:16.658365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26a1bfba-10ca-4a03-ac8d-ca63cffe32ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:16.737318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 503/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:17.005395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c075-296f-46a4-a508-68645825652a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.039670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e98c-9e19-4e2c-99b8-91868faab082 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.040384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e98c-9e19-4e2c-99b8-91868faab082 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.040670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e98c-9e19-4e2c-99b8-91868faab082 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.042448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 504/1023] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:17.048030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f8f9-8685-4faf-9445-d219f67520c3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.048311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f8f9-8685-4faf-9445-d219f67520c3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.048581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82f8f9-8685-4faf-9445-d219f67520c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.067115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9c82f8f9-8685-4faf-9445-d219f67520c3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:17.068335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9c82f8f9-8685-4faf-9445-d219f67520c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:17.071247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.118220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c075-296f-46a4-a508-68645825652a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.140063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 505/1024] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:17.141392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ff518-6758-4d99-a6ba-ddf27be1dace None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.141708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ff518-6758-4d99-a6ba-ddf27be1dace None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.142153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ff518-6758-4d99-a6ba-ddf27be1dace None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.143360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 506/1025] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:17.150829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.151103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.151364 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.152751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.153383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.153684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.153949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.155053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.156482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.161585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 520/1026] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:17.163639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.173610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.174005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:17.175194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:17.241409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 507/1027] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:17.269518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97983967-a748-40ed-be52-62d2d71b434d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.304089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.304708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.304971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.305236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.305932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.307543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.333706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 521/1028] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:51:17 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 19:51:17.405288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.481956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.507271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.515177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44578752-8c16-4f7d-a8a0-cfc092dbe68b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.555139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.555706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.555983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.556623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.566799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:17.571560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:17.572506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:17.573512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 508/1029] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:17.582489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.584405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.628889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880d02c-ac18-41d2-80d0-644261b5f1b1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.666694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.667363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.667644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.667923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.668430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.670196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:17.679046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:17.680037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:17.680985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 522/1030] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:17.686299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.692210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ee6e5-1b85-49bc-983d-003f122add75 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.722336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.741052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.741697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.742303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.742303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.742896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:17.743258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.744640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.753766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 509/1031] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:17.759098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.795874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f70f531-44f8-493f-b741-6a001011a654 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.823944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:17.833806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.834416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.834689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.834954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.835584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] RBAC: Adding query filter params (name=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:17.835876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.837422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.842668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 523/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:17.848970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.874319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a483ec-0ce1-4454-992b-6cbf9138cb2b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.910526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:17.911155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:17.911431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:17.911706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:17.923653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:17.925134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:17.940726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:17.941773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:17.942654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:17.942654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:17.942654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:17.943590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 510/1033] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:17.948271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:17.966701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4563b-c9a9-4330-8254-cb0677d63285 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:17.994026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.005791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.006723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.007004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.007436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.008489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] RBAC: Adding query filter params (type=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:18.009107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.011044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.022293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 524/1034] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:18.023515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.053974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeffb0d-06f0-43a8-b49d-9912366eb31b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.088691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.089204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.089473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.089760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.090402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:18.090689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.092065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.093732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.104103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 511/1035] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:18.109710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.148733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a9da-eb01-40f9-abda-a3174950d8ad None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.185122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.185900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.186351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.186750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.187861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] RBAC: Adding query filter params (interface=public, service_id=e800b3c4b7414d50b0cc69ab2104444e, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:18.188477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.190659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.198501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 525/1036] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/endpoints?service_id=e800b3c4b7414d50b0cc69ab2104444e&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:18.204253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.213631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f024f-55b2-4e21-8eec-26da9f6e3b7a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.217072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.254602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.255240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.255517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.255784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.263316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.264745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:18.276216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:18.277219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:18.278357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 512/1037] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:18.282960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.316666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852de377-3e9b-477d-9bc0-eed2d7c8d4c3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.336406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.355497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.355692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.355876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.356080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.356681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.358092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.364853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 526/1038] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:51:18.388371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69156f-87a0-4cf5-b3f0-da2291764892 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.428954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.429575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.429913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.430158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.431007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:18.431292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.432776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.453621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 513/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:18.459407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.574439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefde91-d03a-4508-9ce6-5d95aea0dc68 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.609972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.610939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.611416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.611892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.663545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.684544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.686840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.788656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 527/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:18.794366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:18.821744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:18.902048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bef140-a022-48b6-a584-18d7c36354a9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:18.937857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:18.938474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:18.938745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:18.939009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:18.949254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:18.950771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:18.965224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:18.966436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:18.967362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:18.967362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:18.967362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:18.967362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 514/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:18.972093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.057344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:19.077290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5b78-5aaa-401f-b129-0cc0f1aacb15 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.111132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.111822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.112169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.112501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.113219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:19.113613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:19.115046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3aeb4-234f-45aa-8096-82273c84dfdb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:19.126522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 528/1042] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:19.132048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.218476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:19.244761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca6b29-7827-4bc5-b36c-c1475466cdb2 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.277994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.278704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.279052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.279381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.286125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:19.287673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:19.289422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855c57c-eede-4145-a556-fa8fd954fcae None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.289831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855c57c-eede-4145-a556-fa8fd954fcae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.290162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855c57c-eede-4145-a556-fa8fd954fcae None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.291726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 529/1043] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:19.297087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.297360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.297655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:19.299770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:19.300638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:19.301579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:19.302701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 515/1044] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:19.306818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.318399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:19.319742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:19.391032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 530/1045] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:19.415116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6adca27-16ce-45ff-842d-cd7613ca62be None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.457042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.457864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.458577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.458951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.459788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:19.460170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:19.461750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:19.463584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:19.481918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 516/1046] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:19.487811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.573872 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:19.599766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fc71d-307a-40b7-9623-550acd3e20a0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.636186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.636792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.637067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.637332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.638086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] RBAC: Adding query filter params (role.id=4ff5a2dfdb484e0ea7887574d77b3025, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:19.638388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:19.639825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:19.645420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 531/1047] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=4ff5a2dfdb484e0ea7887574d77b3025 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:19.660367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.770702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f34024-1a04-4ad8-a40f-5afaba298b72 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.810782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.811554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.811776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.812045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.822833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:19.827529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:19.828466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:19.829335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 517/1048] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:19.834661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:19.943138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848abcd0-c9d1-4c2e-96e5-c4492ae7730e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:19.978687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:19.979287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:19.979553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:19.979809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:19.980475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:19.980768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:19.982210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:19.984646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:19.994974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 532/1049] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:19.999084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.058676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:20.105058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5297485-4aab-4feb-b0a7-e34041ccb59a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.140000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.140906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.141321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.141754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] PATH_INFO: `/v3/users/c1c49454ca214eecaaf4d7553cb720ed` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.156157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] RBAC: Authorizing `identity:get_user(user_id=c1c49454ca214eecaaf4d7553cb720ed)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.157832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c1c2-64ba-4537-843f-a1aff4110546 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.159914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 518/1050] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:19 2022] GET /identity/v3/users/c1c49454ca214eecaaf4d7553cb720ed => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:20.165701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.263012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b14d-54f1-43a0-a908-643027737198 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.263284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b14d-54f1-43a0-a908-643027737198 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.263560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b14d-54f1-43a0-a908-643027737198 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.264928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 519/1051] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:51:20.265956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5a88a-9fee-4a5b-8249-4fdc64e787a1 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.269854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.270140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.270417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.288067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:20.289268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:20.305909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.306833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.307291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.307983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.361135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 520/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:20.376348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:20.380540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.382161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.484864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 533/1053] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:20.490761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.537287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:20.603511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f396aed-2ede-44e0-9511-7eed94d59dde None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.630832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.642162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.642820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.643514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.643794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.645214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.646932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.648743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 521/1054] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:20.654760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.732658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:20.736962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9052f-820b-42c0-90a7-15538b656407 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.764404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391417b7-f205-49e2-ae52-840693b6a21b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.775989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:20.776350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.776972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.777665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.777665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.778152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.779623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:20.788503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:20.789511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:20.790802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 534/1055] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:20.795853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:20.806079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.806676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.806944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.807210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.807932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e, user.id=c1c49454ca214eecaaf4d7553cb720ed) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:20.808233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.809676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.825565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 522/1056] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/role_assignments?user.id=c1c49454ca214eecaaf4d7553cb720ed&scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:20.908759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935158c-bb46-4a36-a30f-943acdca1af9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:20.947152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:20.947779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:20.948055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:20.948322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:20.948962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] RBAC: Adding query filter params (name=image) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:20.949261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:20.950700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:20.960922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 535/1057] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:20.962275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:21.066984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af4c98-ea56-465a-8466-f374f9e7ef00 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:21.101393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:21.102058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:21.102430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:21.102725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:21.103294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] RBAC: Adding query filter params (type=image) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:21.103586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:21.104997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ca8ee-7542-46c8-a91f-94e74f0ecadf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:21.111231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 523/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:21.116869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:21.146156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:21.222112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c872de2-11fe-48c8-9e69-64cdb35d7bba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:21.257400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:21.258425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:21.258941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:21.259424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:21.260300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:21.261938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ce74d-e534-43b1-b40d-e2b679a4de96 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:21.284415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 536/1059] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:51:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:21.522088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd020d0-142e-4b99-8721-f9a3dc572202 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:21.522313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd020d0-142e-4b99-8721-f9a3dc572202 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:21.522599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd020d0-142e-4b99-8721-f9a3dc572202 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:21.524136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 524/1060] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:21.529297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:21.529567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:21.529871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:21.552964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:21.554211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:21.633722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 537/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:21.892633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd020d0-142e-4b99-8721-f9a3dc572202 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:21.936003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:21.997643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd020d0-142e-4b99-8721-f9a3dc572202 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.030287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.030923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.031280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.031566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.042150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:22.047141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:22.048037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:22.048899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:22.053337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 525/1062] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:51:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:22.054597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.065122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:22.155742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fe8f3-a6b7-4b0e-8a42-0a03af750a25 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.190530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.191141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.191417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.191680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.192352 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:22.192648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:22.194058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:22.202320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 538/1063] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:22.207970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.311888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:22.316343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067723a8-3f3f-4e01-ad4a-c7c613c669ec None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.351839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.352494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.352767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.353033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.364355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:22.365947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:22.381903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:22.382951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:22.389153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:22.389153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:22.389153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:22.389153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 526/1064] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:22.390689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.463809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:22.496401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fb6f-1b4b-4919-ba16-ed2464e643b0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.530090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.530688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.530962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.531212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.531866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:22.532159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:22.533549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00220f-904f-44d3-87e4-67063b3d3bb5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:22.548411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 539/1065] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:22.553920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.655824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0234d55-e2f6-4c88-96b4-84d7713d8297 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.690232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.690857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.691129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.691397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.697868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043169e-1ea6-458c-98e9-f6af6583abdc None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.698150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043169e-1ea6-458c-98e9-f6af6583abdc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.698418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043169e-1ea6-458c-98e9-f6af6583abdc None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.699776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 540/1066] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:22.701003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:22.702471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:22.704820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.705310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.705600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:22.714845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:22.715811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:22.716764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:22.721725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 527/1067] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:22.723823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.725680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:22.726867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:22.738719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:22.799693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 541/1068] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:22.830974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231833a-5c9f-4cf4-8a2c-2dee899f4ff3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:22.864218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:22.864829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:22.865106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:22.865376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:22.866136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:22.866427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:22.867857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:22.885359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 528/1069] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:22.891676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:22.951182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:22.992589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32bf805-1a1c-4432-99b0-cdc5b7299dc2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.025974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.026602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.026873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.027133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.037177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.038663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:23.053355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:23.054503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:23.056256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:23.056256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:23.056256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:23.057907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 542/1070] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:23.060591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.061411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.127889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.162501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd028e0-736d-448a-889f-d5f13304fdb9 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.168342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db2aae-155d-4249-a089-dd355406daab None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.183904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.199328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.199942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.200215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.200483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.201124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:23.201419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.202831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e04b9-d689-4ecc-bf18-d36f6e6ee03c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.206101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.206731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.207004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.207271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.214082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 529/1071] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:23.214082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318aac-bd53-4977-bcc6-c751ba6fa879 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.214082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318aac-bd53-4977-bcc6-c751ba6fa879 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.214082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1318aac-bd53-4977-bcc6-c751ba6fa879 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.214082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 530/1072] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:23.219137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:23.221872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1318aac-bd53-4977-bcc6-c751ba6fa879 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b8e2d39c-4eb3-4100-b100-b212fe9bba0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:23.223111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:23.224206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:23.225115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 543/1073] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:23.225842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.225910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.226155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.243343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:23.244462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:23.309451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 544/1074] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:23.310294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.332051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1318aac-bd53-4977-bcc6-c751ba6fa879 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.366909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.376060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.376707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.376984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.377254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.385092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.386574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:23.398049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:23.398955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:23.400141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 531/1075] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:23.405096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.413896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac254c1-5da0-48d0-83e7-7d4439bc6a07 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.416021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.451600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.452207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.452473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.452809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.453385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:23.453709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.455152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.463635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 545/1076] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:23.469305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.512935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a477-a3d7-4f2d-872b-fea3825192b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.554754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.555416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.555703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.555974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.556667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:23.556971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.558430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.579454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 532/1077] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:23.579454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.586825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85f347-6438-449d-9cf2-3d55a61abfcc None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.620265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.620895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.621165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.621435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] PATH_INFO: `/v3/users/c1c49454ca214eecaaf4d7553cb720ed` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.622601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] RBAC: Authorizing `identity:get_user(user_id=c1c49454ca214eecaaf4d7553cb720ed)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.623163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.624043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.625787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 546/1078] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/users/c1c49454ca214eecaaf4d7553cb720ed => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:23.626514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.658144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.674992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48048-92e8-4a8e-8212-2513a0f103a0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.709861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.710471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.710747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.711018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.726425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4530c-5fdd-45c3-bc1f-1532559d358d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.760111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.760761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.761037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.761301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/4ff5a2dfdb484e0ea7887574d77b3025` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.763599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=1244993f093f4ad0920e3a42725a2308, role_id=4ff5a2dfdb484e0ea7887574d77b3025)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.764983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.779764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 547/1079] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:51:23 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/1244993f093f4ad0920e3a42725a2308/roles/4ff5a2dfdb484e0ea7887574d77b3025 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:51:23.780557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.781707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.783178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.839113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.884777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:23.898103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 533/1080] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:23.898314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b686e6-202f-4916-bd49-c33c54102768 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:23.903891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:23.941270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:23.941968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:23.942233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:23.942504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:23.943655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:23.945375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:23.947072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 548/1081] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:23.952984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:24.012282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12465e-4302-404f-a28d-457281f87565 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:24.050275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:24.051029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:24.051225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:24.051613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:24.062732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:24.064189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:24.071580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b66dc-240e-4588-b22c-81ce4478a6a0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:24.073420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:24.082605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:24.083830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:24.084782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:24.084782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:24.084782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:24.084782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 534/1082] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:24.090349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:24.112224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:24.113001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:24.113278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:24.113629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e/users/c1c49454ca214eecaaf4d7553cb720ed/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:24.115744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] RBAC: Authorizing `identity:create_grant(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e, user_id=c1c49454ca214eecaaf4d7553cb720ed, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:24.117324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b0d37-82e6-4510-b48f-924684770a3d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:24.133910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 549/1083] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:51:23 2022] PUT /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e/users/c1c49454ca214eecaaf4d7553cb720ed/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:51:24.198513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c867ba0-af49-43d5-81dc-65e799ad8ea3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:24.237280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:24.238305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:24.238777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:24.239241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:24.240159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:24.240664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:24.242325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b8fcf-681e-44a0-aa04-20683283c7da None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:24.256832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 535/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:25.664066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523602-50d1-4710-9ed4-63745c20c669 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:25.664643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523602-50d1-4710-9ed4-63745c20c669 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:25.664643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39523602-50d1-4710-9ed4-63745c20c669 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:25.666069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 550/1085] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:25.671639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac930d-31e7-48a2-a54b-6e7c8edcf97b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:25.672084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac930d-31e7-48a2-a54b-6e7c8edcf97b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:25.672186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac930d-31e7-48a2-a54b-6e7c8edcf97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:25.691265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-44ac930d-31e7-48a2-a54b-6e7c8edcf97b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:25.692457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-44ac930d-31e7-48a2-a54b-6e7c8edcf97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:25.750692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:25.773977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 536/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:26.011121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ffb1-4bd6-4219-aed9-652dfe325ed6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.011350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ffb1-4bd6-4219-aed9-652dfe325ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.011643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13ffb1-4bd6-4219-aed9-652dfe325ed6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.012987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 551/1087] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:26.018417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.018793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.019070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.036279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13ffb1-4bd6-4219-aed9-652dfe325ed6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.038045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:26.039247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:26.060032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.109884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 537/1088] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:26.143169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13ffb1-4bd6-4219-aed9-652dfe325ed6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.177367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.178013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.178283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.178549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.189612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:26.194343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:26.195299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 17 19:51:26.196229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:26.197728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 552/1089] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:26.201738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.298395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0467664-af07-4ace-8391-04e8a9aa475c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.318115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.333107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.333743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.334015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.334262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.334920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:26.335213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.336562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.344305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 538/1090] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:26.350155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.373175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.431744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.450978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ba755-ecdf-44dc-ad2d-b62208ceacc6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.473128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8e8d-9ebc-4171-997b-66a4ee28c7df None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.483424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.484061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.484365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.484635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.496261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.497885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.506344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.506953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.507251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.507519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:26.512232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:26.513380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:26.514782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:26.514782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:26.514782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:26.514782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 553/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:26.519317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.520269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:26.524943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:26.525990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:26.527360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 539/1092] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:26.532121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.576655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.625062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08214a0e-2355-4323-b715-f57a763e2c0a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.633236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed78ee-2032-4098-9a6f-9a44f98f7c9d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.653327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.659184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.660129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.660399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.660659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.661295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:26.661613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.663023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.668092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.668911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.669268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.669687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.670810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:26.671310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.673143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.676099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 554/1093] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:26.681981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.685624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 540/1094] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:26.691238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.780837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f6d82-533c-47f0-b18e-1473a65b34e9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.789387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68078d8a-ead2-4a27-b253-6a608947cc8e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.791746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.816536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.817169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.817447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.817748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.824983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.826394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:51:26.837727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:26.838686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:26.839660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 555/1095] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:26.841737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.842406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.842690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.842963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] PATH_INFO: `/v3/users/c1c49454ca214eecaaf4d7553cb720ed` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.844165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] RBAC: Authorizing `identity:get_user(user_id=c1c49454ca214eecaaf4d7553cb720ed)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.844593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.845752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.852239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 541/1096] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/users/c1c49454ca214eecaaf4d7553cb720ed => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:26.853295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:26.897163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:26.947706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5759-e5cc-4459-8593-2623d0f36f13 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.956426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b893-478b-4a46-8d17-706cc0e93fb7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:26.982758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.983363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.983644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.983894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:26.984614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:26.984908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:26.986383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:26.989852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:26.990448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:26.990715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:26.990975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.003220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 556/1097] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:27.008936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.032524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.059350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.060791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.106649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e83d10-0e9e-443b-bfcc-1885a2f29ce9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.117300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.141377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.142008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.142277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.142540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.157593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 542/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:27.163633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.218467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.219943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.234895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.266217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b6d53-813b-4301-bbce-6e6379ce118b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.299522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.300168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.300447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.300716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.301872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.303303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.305159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 543/1099] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:27.311143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.322975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 557/1100] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 19:51:27.328623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.339830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.412078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37437dd1-82e9-4d34-afe7-33d4e9c1a60e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.426209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb13188-efa2-417a-989a-814b169b3167 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.446029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.446637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.446913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.447182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.447907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e, user.id=c1c49454ca214eecaaf4d7553cb720ed) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:27.448218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.449667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.455496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 544/1101] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/role_assignments?user.id=c1c49454ca214eecaaf4d7553cb720ed&scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:27.459945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.460596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.460867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.461128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.473028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.474580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:27.491609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:27.492683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:51:27.493706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:27.493706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:51:27.493706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:27.493706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 558/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:27.499259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.503918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.616637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f7530-af51-430e-983c-d561998dcec3 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.651687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.652296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.652589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.652858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.653497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:27.653836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.655238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.667239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 545/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:27.673666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.733927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.774298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef6ac2-875b-4009-8773-bfd4dfe347b5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.808775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.809430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.809704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.809988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.816757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.818246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:51:27.829415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:27.830488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:27.836585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 559/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:27.838642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:27.904937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:27.945053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c8673-97a5-4a7a-a676-46333c5da47e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:27.978291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:27.978893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:27.979164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:27.979429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:27.980115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:27.980421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:27.981874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd81535-27f4-47aa-995f-f8a44e121754 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:27.993319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 546/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:27.999584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:28.108347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88de90f-ed00-44eb-b57a-1de2c36c3534 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:28.131509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:28.144194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:28.144800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:28.145073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:28.145341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:28.146103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] RBAC: Adding query filter params (role.id=4ff5a2dfdb484e0ea7887574d77b3025, scope.project.id=327ecf4949a040998a1557337864548d, user.id=1244993f093f4ad0920e3a42725a2308) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:28.146415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:28.147862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960607db-b5b8-4ff9-86d7-fddda06aa5bb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:28.153732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 560/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:27 2022] GET /identity/v3/role_assignments?user.id=1244993f093f4ad0920e3a42725a2308&scope.project.id=327ecf4949a040998a1557337864548d&role.id=4ff5a2dfdb484e0ea7887574d77b3025 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:29.376403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4d34-aa44-4d99-9060-8f1f2ef53a4c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:29.376965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4d34-aa44-4d99-9060-8f1f2ef53a4c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:29.376965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4d34-aa44-4d99-9060-8f1f2ef53a4c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:29.378365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 547/1107] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:29.384881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:29.385620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:29.385786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:29.416229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:29.417407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:29.492687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 561/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:29.767527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4d34-aa44-4d99-9060-8f1f2ef53a4c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:29.884906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4d34-aa44-4d99-9060-8f1f2ef53a4c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:29.911961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:29.923148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:29.924074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:29.924447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:29.924809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:29.926558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:29.928781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a5a5-5064-4010-b49f-f746bf3fe2ab None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:29.931099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 548/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:29.937412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:30.048973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3f078-4d6c-4765-bfc4-53015f5aa3c8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:30.065319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:30.084633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:30.085229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:30.085501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:30.085794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:30.086440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:30.087915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01a605d-1241-4c48-a22d-38f7a8406714 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:30.112960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 562/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:51:29 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:32.028118 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ef1a-d658-4a04-b838-df9731cbbbe2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:32.028704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ef1a-d658-4a04-b838-df9731cbbbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:32.028704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6ef1a-d658-4a04-b838-df9731cbbbe2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:32.030056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 549/1111] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:32.036300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:32.037007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:32.037170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:32.063863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:32.065050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:32.134833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 563/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:32.410596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ef1a-d658-4a04-b838-df9731cbbbe2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:32.514576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6ef1a-d658-4a04-b838-df9731cbbbe2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:32.551529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:32.552156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:32.552429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:32.552692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] PATH_INFO: `/v3/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:32.554133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] RBAC: Authorizing `identity:get_role(role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:32.555536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:32.556883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 550/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:32 2022] GET /identity/v3/roles/1f37e1386b624e268925de2a48d79cc1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:32.563563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:32.679634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72019862-b21e-40dc-8809-b6a86cbfe76a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:32.681146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:32.715801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:32.716428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:32.716691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:32.716954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] PATH_INFO: `/v3/users/b5d27f439b9f43a485d18c3cc5cc303e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:32.730939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] RBAC: Authorizing `identity:get_user(user_id=b5d27f439b9f43a485d18c3cc5cc303e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:32.732401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:32.734045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 564/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:32 2022] GET /identity/v3/users/b5d27f439b9f43a485d18c3cc5cc303e => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:32.740132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:32.802843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:32.844094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a8939-dc81-45f3-bcfe-9ae022e728c6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:32.877676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:32.878293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:32.878567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:32.878832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:32.947085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:32.948433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:33.006844 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:33.052707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 551/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:33.059702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:33.161792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb2111-ff1b-413b-b811-adc7ff7dfb8b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:33.197262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:33.197847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:33.198124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:33.198384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:33.199546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:33.200984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98a0a2-8b84-4132-8bce-f3e2088ce402 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:33.202608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 565/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:33 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:33.209086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:33.317801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec47dd-d576-49a1-a52b-1d4d028e0bb1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:33.355231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:33.355837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:33.356113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:33.356379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:33.357108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e, user.id=b5d27f439b9f43a485d18c3cc5cc303e) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:33.357401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:33.358863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad1f64-77eb-4a6c-a95e-ab622fd4f298 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:33.364416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 552/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:33 2022] GET /identity/v3/role_assignments?user.id=b5d27f439b9f43a485d18c3cc5cc303e&scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:35.212849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64c9cd-3115-40e6-8f53-3611daafc6eb None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:35.213448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64c9cd-3115-40e6-8f53-3611daafc6eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:35.213448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64c9cd-3115-40e6-8f53-3611daafc6eb None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:35.215549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 566/1118] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:35.223539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:35.223983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:35.224180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:35.242890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:35.244094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:35.261770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:35.318402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 553/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:35.579494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64c9cd-3115-40e6-8f53-3611daafc6eb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:35.611128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:35.690696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64c9cd-3115-40e6-8f53-3611daafc6eb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:35.728039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:35.728657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:35.728925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:35.729197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] PATH_INFO: `/v3/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:35.730658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] RBAC: Authorizing `identity:get_role(role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:35.732038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:35.733337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 567/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:35 2022] GET /identity/v3/roles/1f37e1386b624e268925de2a48d79cc1 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:35.739470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:35.839114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496924a7-a2d6-489b-9ba8-a9ad71279662 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:35.874146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:35.874758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:35.875032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:35.875296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] PATH_INFO: `/v3/users/b5d27f439b9f43a485d18c3cc5cc303e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:35.876536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d27f439b9f43a485d18c3cc5cc303e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:35.877994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:35.879559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 554/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:35 2022] GET /identity/v3/users/b5d27f439b9f43a485d18c3cc5cc303e => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:35.885486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:35.985641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:35.992719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b86ea7-b8db-439e-a7cc-8661459dad3f None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:36.029399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:36.030044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:36.030316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:36.030597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:36.031758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:36.033199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682511e-1ba9-447e-ae5e-712357059c3d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:36.034866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 568/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:35 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:36.041206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:36.076331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:36.157207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52358348-00fd-469e-8909-4bea58ae76b0 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:36.192219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:36.192862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:36.193141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:36.193410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e/users/b5d27f439b9f43a485d18c3cc5cc303e/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:36.196013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e, user_id=b5d27f439b9f43a485d18c3cc5cc303e, role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:36.197444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c95252-aea6-4792-b84a-6dea71eed3a5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:36.217031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 555/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:51:36 2022] PUT /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e/users/b5d27f439b9f43a485d18c3cc5cc303e/roles/1f37e1386b624e268925de2a48d79cc1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:51:38.063394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcfea0-d20f-4228-b887-56a52650f270 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:38.064290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcfea0-d20f-4228-b887-56a52650f270 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:38.064290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcfea0-d20f-4228-b887-56a52650f270 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:38.066445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 569/1124] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:38.072601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:38.073047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:38.073164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:38.097598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:38.098803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:38.169709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 556/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:38.434316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcfea0-d20f-4228-b887-56a52650f270 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:38.543181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcfea0-d20f-4228-b887-56a52650f270 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:38.581006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:38.581649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:38.581920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:38.582187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] PATH_INFO: `/v3/roles/1f37e1386b624e268925de2a48d79cc1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:38.584012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] RBAC: Authorizing `identity:get_role(role_id=1f37e1386b624e268925de2a48d79cc1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:38.585389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:38.586858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 570/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:38 2022] GET /identity/v3/roles/1f37e1386b624e268925de2a48d79cc1 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:51:38.592913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:38.607760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:38.713184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d4d2e-7239-4754-92dc-714047cfbdfa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:38.749580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:38.750228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:38.750500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:38.750773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] PATH_INFO: `/v3/users/b5d27f439b9f43a485d18c3cc5cc303e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:38.752014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] RBAC: Authorizing `identity:get_user(user_id=b5d27f439b9f43a485d18c3cc5cc303e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:38.753444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:38.755078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 557/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:38 2022] GET /identity/v3/users/b5d27f439b9f43a485d18c3cc5cc303e => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:38.761945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:38.798162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:38.870727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63202823-72a3-4cba-a107-10d33dfc85f8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:38.906115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:38.906706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:38.906974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:38.907240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:38.975966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:38.977408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:39.009556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:39.092366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 571/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:39.099230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:39.186701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:39.208258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37748855-7f31-4231-a8a4-2a99266bfc5a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:39.243325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:39.243877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:39.244152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:39.244416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] PATH_INFO: `/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:39.245627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3b02a7a45eb14ff99c2ba22db91ccd9e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:39.247042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b8161-d487-44ba-a794-2b9de93c07e9 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:39.248809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 558/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:39 2022] GET /identity/v3/projects/3b02a7a45eb14ff99c2ba22db91ccd9e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:39.258817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:39.366764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce18cbe-91e6-4d19-96fa-65e40e70aafe None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:39.396785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:39.403433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:39.403982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:39.404280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:39.404548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:39.405273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] RBAC: Adding query filter params (role.id=1f37e1386b624e268925de2a48d79cc1, scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e, user.id=b5d27f439b9f43a485d18c3cc5cc303e) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:39.405579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:39.407030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d058c2-19a1-46b2-80bd-5c5ddfc3490f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:39.412959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 572/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:39 2022] GET /identity/v3/role_assignments?user.id=b5d27f439b9f43a485d18c3cc5cc303e&scope.project.id=3b02a7a45eb14ff99c2ba22db91ccd9e&role.id=1f37e1386b624e268925de2a48d79cc1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:41.369601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9da91-2fd5-4ced-8519-6a90f7aa80d2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:41.370647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9da91-2fd5-4ced-8519-6a90f7aa80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:41.370647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9da91-2fd5-4ced-8519-6a90f7aa80d2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:41.372577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 559/1131] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:41.378912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:41.379597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:41.379748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:41.401452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:41.402679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:41.473503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 573/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:41.739363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9da91-2fd5-4ced-8519-6a90f7aa80d2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:41.858868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9da91-2fd5-4ced-8519-6a90f7aa80d2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:41.893695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:41.894315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:41.894590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:41.894861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:41.895986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:41.897421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a4fdd4-1d83-4a01-91ac-20b70e67d29c None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:41.899104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 560/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:41.906058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:42.020101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84120f5-fde6-4875-97e0-7b6956c1cabf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:42.047112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:42.056798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:42.057397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:42.057666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:42.057926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:42.058555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:42.060044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922d9c7-a27c-499b-800b-1d3593dfc868 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:42.093722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 574/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:51:41 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:44.003526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c4-7196-4ca7-939a-2b88c1cca923 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:44.004113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c4-7196-4ca7-939a-2b88c1cca923 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:44.004113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583c6c4-7196-4ca7-939a-2b88c1cca923 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:44.005401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 561/1135] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:44.011808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:44.012529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:44.012709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:44.037438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:44.038667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:44.110883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 575/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:44.376116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c6c4-7196-4ca7-939a-2b88c1cca923 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:44.397230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:44.495376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583c6c4-7196-4ca7-939a-2b88c1cca923 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:44.536777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:44.537389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:44.537710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:44.537975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:44.539223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:44.540628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b0bc-5dea-40b7-9726-950a68270916 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:44.542356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 562/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:51:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:44.548783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:44.665943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e525f1-c262-49e6-9d43-ab75ff23e4b4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:44.676113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:44.709430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:44.710322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:44.710667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:44.710945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:44.711876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:44.713844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0eca0-31aa-4246-b9bf-8d5f596a24bc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:44.737039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 576/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:51:44 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:51:46.644959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4a501-6956-43f2-a5dc-09070a8b1811 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:46.645849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4a501-6956-43f2-a5dc-09070a8b1811 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:46.645849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4a501-6956-43f2-a5dc-09070a8b1811 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:46.648077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 563/1139] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:46.653461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:46.653935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:46.654044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:46.676306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:46.677512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:46.747434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 577/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:47.015797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4a501-6956-43f2-a5dc-09070a8b1811 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:47.116639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:47.135917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4a501-6956-43f2-a5dc-09070a8b1811 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:47.173040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:47.173681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:47.173959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:47.174225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:47.188658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:47.196333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:47.197490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:47.198686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 564/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:47.206708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:47.311566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:47.323066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c0e6-f6a4-4d57-bb08-e8173d8c1c0a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:47.360103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:47.360704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:47.360975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:47.361241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:47.361935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:47.362232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:47.363611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:47.371923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 578/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:47.381264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:47.490983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8f669-f18e-4015-8aff-4a8cddd9a9c7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:47.526154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:47.526767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:47.527038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:47.527307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] PATH_INFO: `/v3/users/35784dad5c1648ddad81ce0b69592b67` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:47.541376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] RBAC: Authorizing `identity:get_user(user_id=35784dad5c1648ddad81ce0b69592b67)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:47.542834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:47.544563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 565/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:47 2022] GET /identity/v3/users/35784dad5c1648ddad81ce0b69592b67 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:47.554411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:47.666275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc317f2-9365-40cc-9935-92ba6a9cc147 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:47.708353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:47.708955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:47.709227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:47.709490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:47.737321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:47.783897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:47.785365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:47.900965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 579/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:47.907209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:47.910321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:48.009180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a5a3-a85d-49f2-9db7-c8486f6a9638 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:48.044269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:48.044884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:48.045485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:48.045798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] PATH_INFO: `/v3/projects/287ce94130ab4417967afc760068fee9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:48.046963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] RBAC: Authorizing `identity:get_project(project_id=287ce94130ab4417967afc760068fee9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:48.048376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ced2c-0b08-4341-99e4-3b3c0aa8c16d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:48.050027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 566/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:47 2022] GET /identity/v3/projects/287ce94130ab4417967afc760068fee9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:48.056964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:48.122422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:48.170973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d0cd-c6b3-4937-8c9e-449ae79f57fb None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:48.205936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:48.206523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:48.206789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:48.207050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:48.207770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=287ce94130ab4417967afc760068fee9, user.id=35784dad5c1648ddad81ce0b69592b67) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:48.208064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:48.209500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689d7c8-f2df-4aed-9a32-19554182ac06 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:48.214935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 580/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:48 2022] GET /identity/v3/role_assignments?user.id=35784dad5c1648ddad81ce0b69592b67&scope.project.id=287ce94130ab4417967afc760068fee9&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:50.065973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8df0-18cb-4a6f-9268-4af8180a1d76 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:50.066827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8df0-18cb-4a6f-9268-4af8180a1d76 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:50.066827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f8df0-18cb-4a6f-9268-4af8180a1d76 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:50.068975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 567/1147] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:50.076996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:50.077431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:50.077522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:50.097439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:50.098665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:50.173890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 581/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:51:50.437349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8df0-18cb-4a6f-9268-4af8180a1d76 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:50.518952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:50.554076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f8df0-18cb-4a6f-9268-4af8180a1d76 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:50.591048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:50.591669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:50.591942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:50.592213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:50.603083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:50.607776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:50.609578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:50.610686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 568/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:50.615114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:50.667174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:50.719258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e1c13-f7d1-4612-9fd1-36c60c5ea97f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:50.754481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:50.755075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:50.755355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:50.755603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:50.756258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:50.756548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:50.757942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:50.765996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 582/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:50.772473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:50.888234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b22c6-6f22-48f0-919d-d991ce68a534 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:50.917586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:50.925685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:50.926313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:50.926590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:50.926858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] PATH_INFO: `/v3/users/35784dad5c1648ddad81ce0b69592b67` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:50.928095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] RBAC: Authorizing `identity:get_user(user_id=35784dad5c1648ddad81ce0b69592b67)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:50.929526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:50.931137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 569/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:50 2022] GET /identity/v3/users/35784dad5c1648ddad81ce0b69592b67 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:50.940192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:51.044395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34978454-484b-4b4d-b01f-ac668dc342f5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:51.057976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:51.081038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:51.081663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:51.081946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:51.082206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] PATH_INFO: `/v3/projects/287ce94130ab4417967afc760068fee9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:51.083329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] RBAC: Authorizing `identity:get_project(project_id=287ce94130ab4417967afc760068fee9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:51.084883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf3fa-bb82-48e8-95c2-0b076b9ebc12 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:51.086528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 583/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:50 2022] GET /identity/v3/projects/287ce94130ab4417967afc760068fee9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:51.092749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:51.196350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a885fa5-c109-45a6-9530-0e48d0621d04 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:51.231701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:51.232343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:51.232614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:51.232879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] PATH_INFO: `/v3/projects/287ce94130ab4417967afc760068fee9/users/35784dad5c1648ddad81ce0b69592b67/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:51.234970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] RBAC: Authorizing `identity:create_grant(project_id=287ce94130ab4417967afc760068fee9, user_id=35784dad5c1648ddad81ce0b69592b67, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:51.236387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb78f2-ebc1-45d5-aa7d-9e9bae21945f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:51.258237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 570/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:51:51 2022] PUT /identity/v3/projects/287ce94130ab4417967afc760068fee9/users/35784dad5c1648ddad81ce0b69592b67/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:51:53.112913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce51-f6e8-442c-965b-0b33ac1d0885 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:53.113798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce51-f6e8-442c-965b-0b33ac1d0885 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:53.113798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dce51-f6e8-442c-965b-0b33ac1d0885 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:53.115972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 584/1154] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:53.121379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:53.121837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:53.121934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:53.141227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:53.142422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:53.199064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:53.215017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 571/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:53.478215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dce51-f6e8-442c-965b-0b33ac1d0885 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:53.599688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dce51-f6e8-442c-965b-0b33ac1d0885 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:53.634918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:53.635526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:53.635793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:53.636053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:53.647366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:51:53.652026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:53.653165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:51:53.654308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 585/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:51:53.659603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:53.761656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3936b-b81f-469d-ac05-24dc6b861b30 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:53.798491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:53.799081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:53.799359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:53.799620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:53.800272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:53.800568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:53.801946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:53.810332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 572/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:53.816409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:53.861020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:53.926003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be27f0d-6c3a-4ff7-aee5-d7ee1c9501ae None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:53.960808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:53.961416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:53.961718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:53.961980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] PATH_INFO: `/v3/users/35784dad5c1648ddad81ce0b69592b67` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:53.963235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] RBAC: Authorizing `identity:get_user(user_id=35784dad5c1648ddad81ce0b69592b67)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:53.964675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:53.966279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 586/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:51:53 2022] GET /identity/v3/users/35784dad5c1648ddad81ce0b69592b67 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:53.973026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:54.026786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:54.085016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faade7a-6274-4f37-9de3-825d240da089 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:54.119827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:54.120427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:54.120704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:54.120971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:54.188045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:54.189483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:54.238670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:54.292198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 573/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:54.301995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:54.406535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb474d23-be7d-47e1-bffe-0086647511a2 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:54.415369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:54.449030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:54.449690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:54.449968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:54.450240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] PATH_INFO: `/v3/projects/287ce94130ab4417967afc760068fee9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:54.451408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] RBAC: Authorizing `identity:get_project(project_id=287ce94130ab4417967afc760068fee9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:54.452863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04bf6d-93ce-4f2b-a862-88c83fd7424c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:54.454534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 587/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:51:54 2022] GET /identity/v3/projects/287ce94130ab4417967afc760068fee9 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:54.461017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:54.564947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0f567-6941-4631-9448-7a0f9b9a175e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:54.599526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:54.600124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:54.600399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:54.600642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:54.601372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=287ce94130ab4417967afc760068fee9, user.id=35784dad5c1648ddad81ce0b69592b67) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:54.601712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:54.603139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10542ba2-efc0-4653-be11-2486609f8223 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:54.608947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 574/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:51:54 2022] GET /identity/v3/role_assignments?user.id=35784dad5c1648ddad81ce0b69592b67&scope.project.id=287ce94130ab4417967afc760068fee9&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:56.524035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd37f-a739-4c3f-af0f-2d04082a48ce None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:56.524598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd37f-a739-4c3f-af0f-2d04082a48ce None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:56.524598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dd37f-a739-4c3f-af0f-2d04082a48ce None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:56.526086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 588/1162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:56.532732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:56.533446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:56.533606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:56.561126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:56.562348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:56.571972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:56.634685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 575/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:56.909141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd37f-a739-4c3f-af0f-2d04082a48ce None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:57.016879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dd37f-a739-4c3f-af0f-2d04082a48ce None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:57.051275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:57.051864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:57.052134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:57.052388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:57.096858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:57.120434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:57.121938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:57.233183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 589/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:51:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:51:57.239606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:57.338882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c5bbe-0b44-4e97-b461-87347b58a6ee None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:57.373900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:57.374508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:57.374780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:57.375040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:57.381763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:57.383180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4919c086-bcbf-44c5-84ab-8537d0ef26d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:51:57.394476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:51:57.395645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:51:57.396614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 576/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:51:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:51:57.403008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:57.483813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:57.510024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fde91-6097-43b7-9a4e-26a2563cb72a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:57.550549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:57.551154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:57.551475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:57.551828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:57.552786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:51:57.553179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:57.555381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f56befd-f3dc-40e6-a456-cbeab0338596 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:51:57.563291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 590/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 17 19:51:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:51:59.440913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ed49-bb51-40e1-b9f7-ddd8a4dbeb26 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:59.441483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ed49-bb51-40e1-b9f7-ddd8a4dbeb26 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:59.441483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ed49-bb51-40e1-b9f7-ddd8a4dbeb26 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:59.442839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 577/1167] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:51:59.449486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352260-954d-4520-ae37-899a42d93d9f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:59.450219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352260-954d-4520-ae37-899a42d93d9f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:59.450358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352260-954d-4520-ae37-899a42d93d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:59.479618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1f352260-954d-4520-ae37-899a42d93d9f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:51:59.480796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1f352260-954d-4520-ae37-899a42d93d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:51:59.558379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 591/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:51:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:51:59.820944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ed49-bb51-40e1-b9f7-ddd8a4dbeb26 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:51:59.827041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:51:59.931885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ed49-bb51-40e1-b9f7-ddd8a4dbeb26 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:51:59.981685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:51:59.982377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:51:59.982663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:51:59.982932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:51:59.983597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:51:59.984967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352af17-9b03-4770-b4ce-6ca2251360a4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:00.004445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 578/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Aug 17 19:51:59 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:52:01.885253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9b90-bd3e-4c5e-ac73-65a8701d3462 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:01.886252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9b90-bd3e-4c5e-ac73-65a8701d3462 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:01.886252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9b90-bd3e-4c5e-ac73-65a8701d3462 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:01.888306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 592/1170] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:52:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:52:01.896227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:01.896677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:01.896777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:01.916868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:52:01.918107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:01.990788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:02.004129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 579/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:52:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 17 19:52:02.266436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9b90-bd3e-4c5e-ac73-65a8701d3462 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:02.351515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:02.384557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9b90-bd3e-4c5e-ac73-65a8701d3462 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:02.419163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:02.419772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:02.420038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:02.420296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] PATH_INFO: `/v3/domains/c935700bbfff45e286dbcb747c09bfc9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:02.431130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c935700bbfff45e286dbcb747c09bfc9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:02.432578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25791033-a6b5-409b-9b38-16c90725b702 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:02.434154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 593/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 17 19:52:02 2022] GET /identity/v3/domains/c935700bbfff45e286dbcb747c09bfc9 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:02.443532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:02.547420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16fdf-7295-4695-a3a0-7d260422bcee None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:02.583030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:02.583641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:02.583919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:02.584185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:02.584826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:02.586295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db1b3d-9362-4d27-bf47-f1ef6db83549 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:02.613938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 580/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:52:02 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:52:04.507641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947c1c1-3447-4e9e-9b85-9f87792dd54f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:04.508559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947c1c1-3447-4e9e-9b85-9f87792dd54f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:04.508559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947c1c1-3447-4e9e-9b85-9f87792dd54f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:04.510717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 594/1174] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:52:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:52:04.516436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:04.516828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:04.516988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:04.538043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:52:04.539251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:04.585221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:04.614666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 581/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:52:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:52:04.879305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947c1c1-3447-4e9e-9b85-9f87792dd54f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:04.990494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947c1c1-3447-4e9e-9b85-9f87792dd54f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:05.019075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:05.026193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:05.026817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:05.027084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:05.027330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] PATH_INFO: `/v3/domains/c935700bbfff45e286dbcb747c09bfc9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:05.028553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] RBAC: Authorizing `identity:get_domain(domain_id=c935700bbfff45e286dbcb747c09bfc9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:05.030256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5f4f-0f7d-4543-974b-3cd8bcdb267e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:05.031795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 595/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 17 19:52:04 2022] GET /identity/v3/domains/c935700bbfff45e286dbcb747c09bfc9 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:05.038425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:05.144722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b42ce-a364-4652-a976-6f06726c3173 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:05.183247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:05.183858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:05.184137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:05.184406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:05.185036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:05.186514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e28c7c-70be-4cb5-99b6-981e3034b9a6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:05.211815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 582/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:52:05 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:52:07.131939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2815c-1389-4fb6-aec5-9302f4e31540 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:07.132804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2815c-1389-4fb6-aec5-9302f4e31540 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:07.132804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2815c-1389-4fb6-aec5-9302f4e31540 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:07.135033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 596/1178] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:52:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:52:07.141379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:07.142141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:07.142303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:07.165324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:52:07.166544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:07.192706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:07.243297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 583/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:52:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:52:07.508631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2815c-1389-4fb6-aec5-9302f4e31540 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:07.631174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2815c-1389-4fb6-aec5-9302f4e31540 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:07.669864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:07.670863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:07.671137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:07.671400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:07.683572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:52:07.688389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:52:07.689498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:52:07.690598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 597/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:52:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:52:07.697596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:07.800813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bd3c9-d6c3-4ce0-b579-1b88d16a8089 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:07.835661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:07.836262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:07.836537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:07.836806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:07.837481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:52:07.837927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:07.839341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:07.847474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 584/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:52:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:07.853659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:07.914361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:07.958847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827c0ee-9db6-4a13-8c4d-0a325ce0be98 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:07.993319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:07.993979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:07.994271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:07.994537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] PATH_INFO: `/v3/users/46cfee2b88af46b78d1f9d8193e12660` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:08.007752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] RBAC: Authorizing `identity:get_user(user_id=46cfee2b88af46b78d1f9d8193e12660)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:08.009230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:08.010890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 598/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:52:07 2022] GET /identity/v3/users/46cfee2b88af46b78d1f9d8193e12660 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:08.020424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:08.049261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:08.154080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7217f33-bd83-4201-8090-1805b9a3dc1b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:08.191965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:08.192826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:08.193197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:08.193531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:08.265643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:08.267083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:08.295309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:08.375570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 585/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:52:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:52:08.385039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:08.492155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97b7e9-4e20-4a2c-a4e7-bf77c5ed0b0d None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:08.510878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:08.530954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:08.531569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:08.531840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:08.532107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] PATH_INFO: `/v3/projects/7c7f484183ad4756afacad157da1b39e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:08.533245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] RBAC: Authorizing `identity:get_project(project_id=7c7f484183ad4756afacad157da1b39e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:08.534740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b52c0-c0c5-4c0a-828c-43a5d3d51dfc None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:08.536385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 599/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:52:08 2022] GET /identity/v3/projects/7c7f484183ad4756afacad157da1b39e => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:08.542822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:08.644236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3acf6-3c26-485d-be61-850a647b8ce6 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:08.668009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:08.686061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:08.686675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:08.686953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:08.687248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:08.687980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=7c7f484183ad4756afacad157da1b39e, user.id=46cfee2b88af46b78d1f9d8193e12660) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:52:08.688311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:08.689772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5442-e601-48da-a211-97ea4069d45f None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:08.695549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 586/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:52:08 2022] GET /identity/v3/role_assignments?user.id=46cfee2b88af46b78d1f9d8193e12660&scope.project.id=7c7f484183ad4756afacad157da1b39e&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:10.548179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df970b7-e0a9-4a5e-8973-0a4079c8331f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:10.548757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df970b7-e0a9-4a5e-8973-0a4079c8331f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:10.548757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df970b7-e0a9-4a5e-8973-0a4079c8331f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:10.550073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 600/1186] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:52:10.555572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:10.556034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:10.556129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:10.575885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:52:10.577078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:10.649130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 587/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:52:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:52:10.908906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df970b7-e0a9-4a5e-8973-0a4079c8331f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:11.014913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df970b7-e0a9-4a5e-8973-0a4079c8331f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:11.056026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:11.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:11.057146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:11.057410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:11.067799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:52:11.072415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:52:11.073502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:52:11.074569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 601/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:52:11.079811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:11.186001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45027aa8-f0ec-4ccf-a8ac-223695b8041c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:11.207407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:11.232818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:11.233420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:11.233724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:11.233999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:11.234655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:52:11.234959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:11.236312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:11.244355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 588/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:11.251178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:11.313580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:11.377493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d1d12-4f11-438d-b3e0-9efba1ebe3da None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:11.411809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:11.412427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:11.412694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:11.412958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] PATH_INFO: `/v3/users/46cfee2b88af46b78d1f9d8193e12660` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:11.414101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] RBAC: Authorizing `identity:get_user(user_id=46cfee2b88af46b78d1f9d8193e12660)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:11.415548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:11.417137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 602/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:52:11 2022] GET /identity/v3/users/46cfee2b88af46b78d1f9d8193e12660 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:11.426482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:11.538559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645b651-0057-4e20-b328-d0fbfb8e1c9d None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:11.584469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:11.585116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:11.585463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:11.585788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] PATH_INFO: `/v3/projects/7c7f484183ad4756afacad157da1b39e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:11.587547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] RBAC: Authorizing `identity:get_project(project_id=7c7f484183ad4756afacad157da1b39e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:11.589081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806aa17-f88b-420f-b749-faccdc1b804d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:11.590764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 589/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:52:11 2022] GET /identity/v3/projects/7c7f484183ad4756afacad157da1b39e => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:11.596920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:11.707245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f47ea5-72ea-4e7d-978a-b9383fbb26cd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:11.719881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:11.742299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:11.742943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:11.743213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:11.743478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] PATH_INFO: `/v3/projects/7c7f484183ad4756afacad157da1b39e/users/46cfee2b88af46b78d1f9d8193e12660/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:11.745498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=7c7f484183ad4756afacad157da1b39e, user_id=46cfee2b88af46b78d1f9d8193e12660, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:11.746959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277571b0-51a8-4f74-b44a-c4859c7f9f63 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:11.761778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 603/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:52:11 2022] PUT /identity/v3/projects/7c7f484183ad4756afacad157da1b39e/users/46cfee2b88af46b78d1f9d8193e12660/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:52:13.616199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943f61-2a6f-4e8e-ae64-a30808ecfac4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:13.616709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943f61-2a6f-4e8e-ae64-a30808ecfac4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:13.616800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943f61-2a6f-4e8e-ae64-a30808ecfac4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:13.618904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 590/1193] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:52:13.627863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:13.628530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:13.628767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:13.650685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:52:13.651896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:52:13.725755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 604/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:52:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:52:14.003903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943f61-2a6f-4e8e-ae64-a30808ecfac4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:14.061021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:14.129761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943f61-2a6f-4e8e-ae64-a30808ecfac4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:14.165194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:14.165963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:14.166239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:14.166516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:14.177767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:52:14.182400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:52:14.183483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:52:14.184531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 591/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:52:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:52:14.190617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:14.292619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0852e5-b395-4816-948f-d1c23e14417a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:14.294161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:14.331804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:14.332402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:14.332679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:14.332944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:14.333629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:52:14.333930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:14.335313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:14.343502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 605/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:52:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:14.349568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:14.463002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417c015-a4a0-4006-ba41-64e18738e247 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:14.472946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:14.511747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:14.512380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:14.512659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:14.512955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] PATH_INFO: `/v3/users/46cfee2b88af46b78d1f9d8193e12660` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:14.514243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] RBAC: Authorizing `identity:get_user(user_id=46cfee2b88af46b78d1f9d8193e12660)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:14.515686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:14.517287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 592/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 17 19:52:14 2022] GET /identity/v3/users/46cfee2b88af46b78d1f9d8193e12660 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:14.523741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:14.632833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9e61-89ae-4eca-b713-30a35ad4ce12 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:14.667996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:14.668588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:14.668857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:14.669119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:14.701069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:14.742074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:14.743527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:14.862848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 606/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:52:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:52:14.869421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:14.975460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5395835-21af-4f83-aad4-3e6abea88619 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:15.010084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:15.010700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:15.010981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:15.011254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] PATH_INFO: `/v3/projects/7c7f484183ad4756afacad157da1b39e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:15.012489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] RBAC: Authorizing `identity:get_project(project_id=7c7f484183ad4756afacad157da1b39e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:15.013969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2ea38-44c7-451d-9b5b-59354fb59fdb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:15.015595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 593/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 17 19:52:14 2022] GET /identity/v3/projects/7c7f484183ad4756afacad157da1b39e => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:15.022745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:52:15.100211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:15.142644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf50a0-f262-447a-b0f3-73f1c85b25a6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:52:15.178614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:52:15.179205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:52:15.179474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:52:15.179735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:52:15.180576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=7c7f484183ad4756afacad157da1b39e, user.id=46cfee2b88af46b78d1f9d8193e12660) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:52:15.180881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:52:15.182408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f1faf-0848-4520-a76a-29397fd4ebd0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:52:15.188896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 607/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:52:15 2022] GET /identity/v3/role_assignments?user.id=46cfee2b88af46b78d1f9d8193e12660&scope.project.id=7c7f484183ad4756afacad157da1b39e&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:52:25.005354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=78 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:52:25.185834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=36 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:10.457984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d69760-f3a7-4e60-b936-986c0a44dd2f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:10.458631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d69760-f3a7-4e60-b936-986c0a44dd2f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:10.458631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d69760-f3a7-4e60-b936-986c0a44dd2f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:10.460248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 594/1201] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:10.466000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:10.466544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:10.466625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:10.487119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:10.488394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:10.593884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 608/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:10.879546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d69760-f3a7-4e60-b936-986c0a44dd2f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:11.034557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d69760-f3a7-4e60-b936-986c0a44dd2f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:11.074351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:11.075237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:11.075638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:11.075882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:11.087017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:11.088451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:11.104473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:11.106147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:11.107137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:11.107137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:11.107137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:11.112904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 595/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:11.113074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:11.251540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108a90a-3f0e-4fae-8401-97b18bca3e90 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:11.289089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:11.297518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:11.298434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:11.298750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:11.299049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:11.299769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:11.300184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:11.301690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2a242-31ea-4d47-b6bc-8f16b487484d None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:11.314315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 609/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:11.319727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:11.379789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:11.438090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9682c-721b-44ef-8f08-ec89e442c4fd None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:11.477039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:11.478704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:11.479129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:11.479465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:11.480299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:11.481915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695d7cd-1101-4b01-a1cb-f835df9d9f76 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:11.522135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 596/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 17 19:53:11 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:53:13.533281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44209-50fc-4ba7-89bb-1e1f38ad5f16 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:13.533887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44209-50fc-4ba7-89bb-1e1f38ad5f16 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:13.534104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44209-50fc-4ba7-89bb-1e1f38ad5f16 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:13.535830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 610/1206] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:13.541132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:13.541577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:13.541790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:13.560409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:13.561629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:13.632215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 597/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:13.898714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c44209-50fc-4ba7-89bb-1e1f38ad5f16 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.014919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c44209-50fc-4ba7-89bb-1e1f38ad5f16 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:14.055578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:14.056494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:14.056765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:14.057028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:14.069523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:14.074364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:14.075466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:14.081625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 611/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:14.084637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.130128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:14.195172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680eeb0-ca97-4be4-8f3c-460966965705 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:14.229810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:14.230407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:14.230685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:14.231020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:14.231690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:14.231977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:14.234147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:14.245501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 598/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:53:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:14.250656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.310580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:14.379123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2236b59-46c9-4d0f-89ee-c59178c7e074 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:14.419850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:14.420848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:14.421494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:14.422368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:14.437294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:14.439406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:14.458106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:14.459264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:14.460758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:14.460758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:14.460758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:14.460758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 612/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:14.465475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.578320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:14.582334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac40bc-cf1f-4d09-b259-5f6475a417a8 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:14.618397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:14.619165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:14.619539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:14.619903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:14.620644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:14.621038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:14.622751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:14.636182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 599/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:14.640853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.759344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ba57-59ae-41ab-a28e-40d5f8f3b244 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:14.761211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:14.802509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:14.803202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:14.803627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:14.803986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:14.814485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:14.816254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:14.830334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:14.831485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:14.832470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 613/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:14.840394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:14.969925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3e5f2-b6b5-44be-8867-43ac37e97cc1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:15.012074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:15.012814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:15.013149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:15.013512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:15.014440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:15.014763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:15.016393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:15.042936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 600/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:15.044949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:15.166365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9fed1-39d7-4629-a0fa-8c806cd100b9 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:15.207568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:15.208187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:15.208459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:15.208717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:15.242855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:15.284004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:15.285390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:15.396265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 614/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:53:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:53:15.402160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:15.423876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:15.509886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209862d-bc05-4f1d-b2a5-76d5ee514372 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:15.545098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:15.545846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:15.546223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:15.546913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:15.557892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:15.559720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:15.574906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:15.576075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:15.577281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:15.577281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:15.577281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:15.583018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:15.585259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 601/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:15.650614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:15.692713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08a273-4a7f-4a71-8eba-5a846df1cca6 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:15.732040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:15.732653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:15.732929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:15.733200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:15.733879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:15.734172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:15.735595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:15.748282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 615/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:15.754529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:15.845672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:15.880496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bc43-f184-4c0a-87de-8079c5e6f900 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:15.918515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:15.919323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:15.919701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:15.920076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:15.928169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:15.929912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:15.943871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:15.945050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:15.946199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:15.947718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 602/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:15.953822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:16.072220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bbcc-dac9-42db-817e-ef5078b883e0 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:16.101166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:16.115903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:16.116812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:16.117236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:16.117684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:16.118566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:16.119063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:16.120824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad3ef4-4629-4ac2-9546-76ef5c18d90b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:16.141190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 616/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:16.143341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:16.251546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fc9f1-6e7f-4a29-bffc-7d957b994d70 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:16.279947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:16.286558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:16.287254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:16.287672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:16.288009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:16.288808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=7abc4e612c0241ac8efb2789a5741250) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:16.289181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:16.290758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38659e3a-d8f2-4f62-9056-6f6a94155355 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:16.305643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 603/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:53:16 2022] GET /identity/v3/role_assignments?user.id=7abc4e612c0241ac8efb2789a5741250&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:18.192491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04757a1-c510-4a1e-b126-fbaf7e27296d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:18.193069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04757a1-c510-4a1e-b126-fbaf7e27296d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:18.193069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04757a1-c510-4a1e-b126-fbaf7e27296d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:18.194379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 617/1220] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:18.199765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:18.200197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:18.200519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:18.219889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:18.221077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:18.293091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 604/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:18.554416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04757a1-c510-4a1e-b126-fbaf7e27296d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:18.658800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04757a1-c510-4a1e-b126-fbaf7e27296d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:18.694121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:18.694753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:18.695026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:18.695297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:18.707084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:18.711687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:18.712663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:18.713724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 618/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:18.719587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:18.822543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34fff4-afca-4f5e-9b34-28045830d9b6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:18.847410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:18.859859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:18.860478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:18.860741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:18.861005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:18.861716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:18.862024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:18.863401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:18.871516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 605/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:53:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:18.878652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:18.928653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:18.990294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f0f0c-e54d-423d-973c-2b08a2729e5b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.028374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.029064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.029333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.029628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.039562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.041488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:19.055732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:19.056875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:19.058067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:19.058067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:19.058067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:19.058067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 619/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:19.064412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.179780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d377a08-7675-40ba-bc41-be680ffc4b10 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.212758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.213486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.213676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.213940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.214606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:19.214902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.216502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.228433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 606/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:19.234744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.330323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:19.338579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d0d55-0aef-4c6c-9c39-aa207380dec4 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.382105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.382853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.383280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.383640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.392956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.394456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:19.406193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:19.407467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:19.408718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 620/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:19.414934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.469079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:19.525441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94237108-512f-4140-8963-cf4fa1225b81 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.567689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.568532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.569016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.569503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.570385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:19.570754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.572359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.592040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 607/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:19.598414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.710147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf704d3-4874-491b-a3c4-364ee9c617d8 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.744617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.745229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.745549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.745950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.757237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.758866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:19.773073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:19.779740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:19.780936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:19.780936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:19.780936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:19.780936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 621/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:19.781212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.886795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b245b33-2913-4fd0-95cf-45d68fb2e3dc None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:19.889374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:19.923757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:19.924420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:19.924707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:19.925003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:19.925839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:19.926131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:19.927591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:19.939952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 608/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:19.946310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:19.956705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:20.052241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83245b-bc56-4741-a322-ab4055bd7e49 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:20.088091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:20.089198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:20.089460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:20.089830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:20.098554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:20.100162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:20.112319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:20.113426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:20.114367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 622/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:20.119467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:20.227123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188353b-ef8b-4234-aa88-1fb1f2d8b22a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:20.261731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:20.262410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:20.262682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:20.262951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:20.263791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:20.264092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:20.265519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a083d0-1b3e-4a61-b6b3-2ebd77ef8838 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:20.277836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 609/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:20.284101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:20.354602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:20.390849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7523e6-217f-4a3b-a8c9-d6d53014ed7a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:20.426571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:20.427325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:20.427597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:20.427877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/7abc4e612c0241ac8efb2789a5741250/roles/c55245851594417e937d6b554d67ca31` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:20.442976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=7abc4e612c0241ac8efb2789a5741250, role_id=c55245851594417e937d6b554d67ca31)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:20.444405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d640f5-e015-49bf-92f3-191800ab172c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:20.457410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 623/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:53:20 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/7abc4e612c0241ac8efb2789a5741250/roles/c55245851594417e937d6b554d67ca31 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:53:22.415937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6ed4-7f6e-45ca-b3bc-12f6e17213e1 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:22.416553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6ed4-7f6e-45ca-b3bc-12f6e17213e1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:22.416553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa6ed4-7f6e-45ca-b3bc-12f6e17213e1 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:22.417852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 610/1233] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:22.423934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:22.424444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:22.424444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:22.443246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:22.444483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:22.514458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 624/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:22.780788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa6ed4-7f6e-45ca-b3bc-12f6e17213e1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:22.807167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:22.900498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa6ed4-7f6e-45ca-b3bc-12f6e17213e1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:22.939165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:22.939800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:22.940079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:22.940346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:22.952916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:22.958016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:22.959027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 17 19:53:22.959999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:22.962978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 611/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 17 19:53:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:22.966246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.018552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:23.080554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e330a2c-ccb9-4a9e-8daf-d445e2f3481e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.119905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.120579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.120951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.121222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:23.121947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] RBAC: Adding query filter params (name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:23.122245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:23.123680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:23.132929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 625/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 17 19:53:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:23.139169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.230297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:23.247417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfc89d-76d8-4a12-9758-4ec9e3528f89 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.281964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.282592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.282868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.283122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:23.294101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:23.295807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:23.311309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:23.312471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:23.318488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:23.318488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:23.318488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:23.318488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 612/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:23.318963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.428825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b078625-5ece-4b16-bb4b-86eca085e155 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.439155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:23.467885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.468525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.468789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.469066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:23.469718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:23.470025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:23.471400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:23.484314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 626/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:53:23.490585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.594727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263da38-97ad-42c4-9614-6334de434a2a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.632238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.632871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.633141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.633398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:23.636291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:23.642825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:23.644268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:23.658050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:23.659287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:23.660496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 613/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:23.665307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.786187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00106e-8a22-4c40-8ca1-7d731254343b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.823001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.823607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.823880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.824144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:23.824910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:23.825380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:23.826851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:23.845797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 627/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:23.851830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:23.960254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f0ba8-b6a1-4447-960d-525d9e0dd536 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:23.996101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:23.996923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:23.997224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:23.997582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:24.065138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:24.072880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:24.074476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:24.175587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 614/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:53:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 17 19:53:24.181800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:24.210482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:24.288026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb134bd-9a6d-4e1d-9f91-794aba01efe5 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:24.325164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:24.325971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:24.326245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:24.326585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:24.336646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:24.338112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:24.352366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:24.353424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:24.354772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:24.354772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:24.354772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:24.354772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 628/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:24.359753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:24.463229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:24.469993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6bc79-73fa-4bff-a1e8-ac838db18b37 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:24.505375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:24.506119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:24.506267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:24.506530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:24.507163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:24.507462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:24.508860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:24.520379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 615/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:24.526755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:24.635662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:24.659429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c1660-25f2-45f5-be8f-d48dee174d79 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:24.699938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:24.700952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:24.701414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:24.702108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:24.711695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:24.713524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:24.728354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:24.729725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:24.730964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 629/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:24.736336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:24.842303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadefcd4-b52f-4a29-bb07-0df92b0585ba None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:24.879845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:24.880455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:24.880729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:24.881007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:24.881744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:24.882039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:24.883463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ce94a-85ad-47ee-9e78-a6e8d99df270 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:24.895610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 616/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:24.904102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:25.020958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d81fc2-e786-4f14-acdc-3a580bb97454 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:25.058240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:25.058905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:25.059191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:25.059478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:25.060208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] RBAC: Adding query filter params (role.id=c55245851594417e937d6b554d67ca31, scope.project.id=327ecf4949a040998a1557337864548d, user.id=7abc4e612c0241ac8efb2789a5741250) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:25.060497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:25.062378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02b887-41c0-4a7d-89b5-0e6bb6d2e56e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:25.068444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 630/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:53:24 2022] GET /identity/v3/role_assignments?user.id=7abc4e612c0241ac8efb2789a5741250&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c55245851594417e937d6b554d67ca31 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:27.022639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7b47-8960-4497-9dc6-5a946ee18d44 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.023216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7b47-8960-4497-9dc6-5a946ee18d44 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.023216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a7b47-8960-4497-9dc6-5a946ee18d44 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.024726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 617/1247] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:27.030073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.030481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.030657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.037246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:27.051577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:27.052853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:27.127440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 631/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:27.397309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a7b47-8960-4497-9dc6-5a946ee18d44 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:27.419074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:27.510456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a7b47-8960-4497-9dc6-5a946ee18d44 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:27.548582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:27.549235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.549499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.549806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.563837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:27.568518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:27.569694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:27.570576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 618/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:53:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:27.576170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:27.681206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812632c9-a194-4ebd-91be-aebead3c94ae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:27.714691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:27.721276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:27.722034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.722348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.722623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.723413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:27.723724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:27.725310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:27.734681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 632/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:53:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:27.740669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:27.825415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:27.849320 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4836c-626e-4ffb-a3f3-9e417320a67e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:27.885550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:27.886241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:27.886514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:27.886779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:27.897187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:27.898947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:27.914692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:27.924720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:27.926291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:27.926291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:27.926291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:27.926291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 619/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:27.926700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.038879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ef95-c8e0-4d91-848d-bfe2aeb5a238 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.078047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.078748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.078992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.079276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:28.080022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:28.080350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:28.082010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:28.094973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 633/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:53:28.102224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.211348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccce396-4d3b-46cb-b358-def068b8a60b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.240249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:28.248865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.250462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.250748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.251018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:28.259502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:28.260934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:28.272953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:28.274308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:28.275379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 620/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:28.280672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.351071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:28.402962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d826d5-078c-4fce-bb48-56d5abb95f33 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.446016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.446662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.447171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.447449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:28.448198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:28.448503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:28.450289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:28.470729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 634/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:28.477484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.594768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41df72-fc2f-4d18-a63f-f6bdde30e8b4 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.636731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163006b-0035-4044-a643-7ef522190273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.637502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163006b-0035-4044-a643-7ef522190273 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.638022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163006b-0035-4044-a643-7ef522190273 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.638358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163006b-0035-4044-a643-7ef522190273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:28.719736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:28.735465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163006b-0035-4044-a643-7ef522190273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:28.736938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163006b-0035-4044-a643-7ef522190273 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:28.845842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 621/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:53:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:53:28.852253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:28.959799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c883f-9656-4c06-8ab8-d044f7280081 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:28.974414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:28.995986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:28.996653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:28.996946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:28.997160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:29.009627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:29.011153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:29.028295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:29.029373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:29.030542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:29.030542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:29.030542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:29.030542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 635/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:29.036386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163006b-0035-4044-a643-7ef522190273 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:29.133347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:29.147630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163006b-0035-4044-a643-7ef522190273 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:29.185437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:29.186075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:29.186379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:29.186630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:29.187247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:29.187542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:29.188913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:29.200999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 622/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:29.207255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:29.323253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718f417-8a69-474a-af33-bad0b34f6c0a None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:29.359944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:29.360581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:29.360855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:29.361116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:29.368512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:29.370021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:29.382590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:29.383929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:29.385167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 636/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:29.390189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:29.491345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f4d17-1059-4d0a-a70e-ee49f5701dca None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:29.526421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:29.527068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:29.527319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:29.527598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:29.528315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:29.528622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:29.530086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a831b265-0d70-47bd-95ab-c166f3ad5023 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:29.544831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 623/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:29.551403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:29.580068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:29.668357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe53bbf-f6b0-4a39-995c-3bfa8bf096ca None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:29.708201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:29.709155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:29.709491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:29.709880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:29.711167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=7abc4e612c0241ac8efb2789a5741250) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:29.711560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:29.713966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642b1fa-8a8e-41df-b663-e7810beb72ef None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:29.721464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 637/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:53:29 2022] GET /identity/v3/role_assignments?user.id=7abc4e612c0241ac8efb2789a5741250&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:31.602402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0e6c-a89c-48b1-9e6c-a557a123c79d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:31.603204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0e6c-a89c-48b1-9e6c-a557a123c79d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:31.603204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0e6c-a89c-48b1-9e6c-a557a123c79d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:31.605461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 624/1261] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:31.611828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:31.612569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:31.612661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:31.640233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:31.641420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:31.720924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 638/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:31.992814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db0e6c-a89c-48b1-9e6c-a557a123c79d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.005725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:32.111638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db0e6c-a89c-48b1-9e6c-a557a123c79d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:32.147037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:32.147664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:32.147938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:32.148201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:32.159167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:32.163830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:32.164881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:32.165885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 625/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:32.171301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.191092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:32.280894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903803-8e4c-4640-be31-b8eca1dd3da3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:32.316378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:32.316974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:32.317210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:32.317495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:32.318188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:32.318481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:32.319856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:32.328016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 639/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:32.334826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.410209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:32.448694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fbc5d-ee25-4713-80a3-b47327234609 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:32.485332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:32.485974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:32.486249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:32.486521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:32.497420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:32.498885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:32.513256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:32.514390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:32.515469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:32.515469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:32.515469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:32.515469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 626/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:32.522832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.609493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:32.631942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d4468-d911-4c6a-ab97-e0b6e7615cdd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:32.668645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:32.669245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:32.669514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:32.670864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:32.670864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:32.671771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:32.673522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:32.688051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 640/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:32.694375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.801336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc847-c2b8-4228-9557-e4c8598edc4a None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:32.821017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:32.835899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:32.836536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:32.836809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:32.837077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:32.845459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:32.846938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:32.857847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:32.858857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:32.859889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:32.861706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 627/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:32.865053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:32.968292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ef0f-0723-4ee3-b7d4-bd5f3033252e None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.009776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.010378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.010645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.010909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.011632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:33.011929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.013477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.022294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:33.031947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 641/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:33.040670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:33.149463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f1834-762c-4394-b9ee-5b6ec5ca1e41 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.185660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.186269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.186537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.186802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.197443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.198917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:33.212934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:33.214052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:33.214984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:33.214984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:33.214984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:33.214984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 628/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:33.220212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:33.335386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1f523-6274-43a8-abd9-d22541b3faba None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.369169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.369781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.370048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.370306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.370935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:33.371231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.372688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.383456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 642/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:33.389274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:33.423819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:33.491225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1401b2e-07f7-4c07-8be9-b3f1bccb8be1 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.524395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.525003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.525277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.525562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.532445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.533896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:33.544953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:33.545993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:33.547398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 629/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:33.552538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:33.611451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:33.663728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74eb8a-c141-404b-94a3-74c7c5a97e69 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.710005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.710526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.710819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.711124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.711828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:33.712131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.713602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca77773-aac5-489f-8971-6b1038dcca21 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.726299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 643/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:53:33.733194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:33.828234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:33.843184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dcf95-40b8-45f8-9233-5ad3fa8a5390 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:33.883223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:33.883860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:33.884129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:33.884396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] PATH_INFO: `/v3/projects/327ecf4949a040998a1557337864548d/users/7abc4e612c0241ac8efb2789a5741250/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:33.886372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] RBAC: Authorizing `identity:create_grant(project_id=327ecf4949a040998a1557337864548d, user_id=7abc4e612c0241ac8efb2789a5741250, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:33.888035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a90fd-e588-4251-9c40-b48befc06953 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:33.908375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 630/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 17 19:53:33 2022] PUT /identity/v3/projects/327ecf4949a040998a1557337864548d/users/7abc4e612c0241ac8efb2789a5741250/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:53:35.775878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564592b-999f-446e-84dd-08dccbfd2dd2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:35.776453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564592b-999f-446e-84dd-08dccbfd2dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:35.776453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564592b-999f-446e-84dd-08dccbfd2dd2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:35.777827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 644/1274] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:35.783268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:35.783680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:35.783787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:35.804030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:35.805215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:35.877948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 631/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:36.143201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564592b-999f-446e-84dd-08dccbfd2dd2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:36.260430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564592b-999f-446e-84dd-08dccbfd2dd2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:36.278109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:36.299363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:36.299981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:36.300281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:36.300518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:36.312000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 17 19:53:36.316663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:36.317818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:36.318765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 645/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 17 19:53:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:36.324015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:36.410332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:36.433032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92e15f-1523-42fe-9b10-0c0c0d7ab9c9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:36.468382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:36.468976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:36.469249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:36.469516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:36.470210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:36.470514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:36.471845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:36.479902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 632/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 17 19:53:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:36.489169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:36.596509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682308fc-f748-4432-99da-53c876cf3440 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:36.632253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:36.632871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:36.633142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:36.633410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:36.643895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:36.645355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:36.660190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:36.661334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:36.663259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:36.663259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:36.663259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:36.663259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 646/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:36.670140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:36.792928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19540d53-7523-4c98-b775-99f06e4dafab None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:36.827922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:36.828512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:36.828784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:36.829025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:36.829714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:36.830017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:36.831371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:36.843242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 633/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:36.849162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:36.882298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:36.956306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33386be9-d534-4bb4-99c9-e47eb7b84666 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:36.991360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:36.991965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:36.992232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:36.992494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:37.000343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:37.001817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 17 19:53:37.013842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:37.014910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:37.016081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 647/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:37.022005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:37.028125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:37.131209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067575f3-9ca5-4db0-a2a2-a05caa7d4a1e None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:37.165779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:37.166375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:37.166645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:37.166916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:37.167637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:37.167936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:37.169377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:37.187652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 634/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 17 19:53:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:37.196890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:37.293661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:37.310201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d3e88-39ff-4320-adcb-3b098edf1192 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:37.343700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:37.344293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:37.344564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:37.344829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:37.415903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:37.417350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:37.513095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:37.524370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 648/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 17 19:53:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 17 19:53:37.530407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:37.598511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:37.637298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1542c-9173-40c2-bddc-6d1e86311ed7 None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:37.672389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:37.672945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:37.673233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:37.673486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:37.684551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:37.686677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 17 19:53:37.706363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:37.707463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 17 19:53:37.708820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 17 19:53:37.708820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 17 19:53:37.708820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:37.708820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 635/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:37.713463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:37.836459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0287fe-a51c-4780-bdb8-71fbc599c91c None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:37.871672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:37.872258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:37.872513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:37.872772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:37.873408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:37.873735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:37.875082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:37.887284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 649/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 17 19:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:37.896460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:38.004955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9117b-e916-481b-9b82-d033d0104ceb None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:38.018696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:38.045599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:38.046228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:38.046503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:38.046767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:38.054061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:38.055489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 17 19:53:38.067671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:38.068732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: ERROR keystone.server.flask.application Aug 17 19:53:38.070010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 636/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 17 19:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:38.074917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:38.127330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:38.186940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f938ba2-a388-4407-bc7a-4d6856813dd3 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:38.223070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:38.223663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:38.223936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:38.224202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:38.224889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:38.225185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:38.226661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea118ff-fac9-43cb-84b7-3dc9f893bf76 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:38.239273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 650/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 17 19:53:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:38.248351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:38.374980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f428e-4004-4878-bcd5-42945fc63bdf None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:38.410315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:38.410843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:38.411112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:38.411375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:38.412090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] RBAC: Adding query filter params (role.id=c974a286ee1942c8bab82b232c6af142, scope.project.id=327ecf4949a040998a1557337864548d, user.id=7abc4e612c0241ac8efb2789a5741250) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:38.412396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:38.413857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9463f-90a7-4e7f-a48c-de6f5dcdd272 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:38.419757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 637/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 17 19:53:38 2022] GET /identity/v3/role_assignments?user.id=7abc4e612c0241ac8efb2789a5741250&scope.project.id=327ecf4949a040998a1557337864548d&role.id=c974a286ee1942c8bab82b232c6af142 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:40.308407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8fd88-39a6-445a-ae27-cc650de810e1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:40.309450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8fd88-39a6-445a-ae27-cc650de810e1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:40.309450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8fd88-39a6-445a-ae27-cc650de810e1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:40.311625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 651/1288] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:40.320065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:40.320474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:40.320665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:40.346873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:40.348095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:40.379010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:40.418488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 638/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:40.695214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8fd88-39a6-445a-ae27-cc650de810e1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:40.810604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8fd88-39a6-445a-ae27-cc650de810e1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:40.821197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:40.854519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:40.855140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:40.855412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:40.855681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:40.856266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:40.857673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:40.867010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:40.868244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:40.869453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 652/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:53:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 17 19:53:40.874338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:40.981536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb89a-be6d-499d-ac0a-f735e0fe32c0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:41.022174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:41.022785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:41.023063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:41.023332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:41.023971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:41.024276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:41.025701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cdbc5f-a532-4702-ae7d-f7f1ff12bfc8 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:41.031533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 639/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 17 19:53:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:41.040803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:41.162761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f350c3-c566-4483-8d0f-db8ba503f39b None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:41.204815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:41.205414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:41.205723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:41.205983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] PATH_INFO: `/v3/services` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:41.206609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:41.206903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:41.208299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b91bf9-3c4f-4ac1-963c-204bb1ffe907 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:41.213224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 653/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 17 19:53:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:43.060947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc9c4-7d57-4d04-905e-ee3dd56892eb None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:43.061903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc9c4-7d57-4d04-905e-ee3dd56892eb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:43.061903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc9c4-7d57-4d04-905e-ee3dd56892eb None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:43.064066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 640/1293] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:43.070500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c22469-c071-4f4b-b71c-bd64d968c578 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:43.070940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c22469-c071-4f4b-b71c-bd64d968c578 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:43.071123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c22469-c071-4f4b-b71c-bd64d968c578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:43.098132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-17c22469-c071-4f4b-b71c-bd64d968c578 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:43.099332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-17c22469-c071-4f4b-b71c-bd64d968c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:43.135794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:43.174614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 654/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:43.442470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fc9c4-7d57-4d04-905e-ee3dd56892eb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:43.483338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:43.570411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fc9c4-7d57-4d04-905e-ee3dd56892eb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:43.606107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:43.606701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:43.606980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:43.607249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:43.607894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:43.609242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b8438-b227-46b0-8da7-466e54a3b1d0 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:43.625979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 641/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 17 19:53:43 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:53:45.512376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cf88-defa-490b-bf65-b6f15efa5385 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:45.512941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cf88-defa-490b-bf65-b6f15efa5385 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:45.512941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938cf88-defa-490b-bf65-b6f15efa5385 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:45.514311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 655/1296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:53:45.520662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:45.521405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:45.521581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:45.542927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:45.544145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:45.620337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 642/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:45.884340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938cf88-defa-490b-bf65-b6f15efa5385 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:45.996404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938cf88-defa-490b-bf65-b6f15efa5385 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:46.031537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:46.032136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:46.032403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:46.032663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:46.033237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:46.034637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:46.043448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:46.044595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:46.045651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 656/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:53:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:46.051637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:46.073023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:46.160349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c64a3-e06c-472e-874e-6bd46c045159 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:46.200041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:46.200630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:46.200903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:46.201172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:46.201827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:46.202128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:46.203476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:46.209162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 643/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 17 19:53:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:46.217862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:46.254689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:46.328212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd086-1e01-44ff-9da7-b4da4cda6a19 None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:46.363376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:46.363978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:46.364254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:46.364524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:46.365224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] RBAC: Adding query filter params (interface=public, service_id=84c5a048c623462b82c27d720baaf2f9, region_id=RegionOne) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:46.365526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:46.366942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b8555-298d-4196-a73e-e48d2fe065fd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:46.372087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 657/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 17 19:53:46 2022] GET /identity/v3/endpoints?service_id=84c5a048c623462b82c27d720baaf2f9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:53:46.378861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:46.464012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:46.484614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1596-00c4-4007-96e1-01ac752c323b None admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:46.521348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:46.522022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:46.522229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:46.522498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:46.523100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:46.524446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d11a6b-b97d-4b7e-be5f-a7097225184d None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:46.531646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 644/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 17 19:53:46 2022] GET /identity/v3/services => generated 2191 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:53:48.405376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f1b2-ba21-4595-9c46-9da1aa599858 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:48.406017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f1b2-ba21-4595-9c46-9da1aa599858 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:48.406017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61f1b2-ba21-4595-9c46-9da1aa599858 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:48.407280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 658/1302] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:53:48.413594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:48.414327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:48.414460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:48.441277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:53:48.442501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:53:48.516667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 645/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:53:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 17 19:53:48.785904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61f1b2-ba21-4595-9c46-9da1aa599858 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:48.901878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:48.911259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61f1b2-ba21-4595-9c46-9da1aa599858 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:48.946439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:48.947049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:48.947323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:48.947583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:48.948163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:48.949552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 17 19:53:48.958587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 17 19:53:48.959734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.get_or_create( Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application with Lock( Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self._enter() Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application return self.creator() Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application created_value = creator( Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: ERROR keystone.server.flask.application Aug 17 19:53:48.960847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 659/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 17 19:53:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 17 19:53:48.966124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:49.048082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:49.076825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0daee-f922-476f-b716-040e1dee26b9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:49.112042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:49.112635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:49.112906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:49.113171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] PATH_INFO: `/v3/services` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:49.113824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:53:49.114121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:49.115474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8354e92-a255-4f6a-8e12-327518265844 None admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:49.120671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 646/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 17 19:53:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 19:53:49.127502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:53:49.246488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27699c-df0c-4de3-ab33-e7ccd91ee8bd None admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:53:49.281678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:53:49.282269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:53:49.282545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:53:49.282806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:53:49.283462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:53:49.284840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d0ee7-4450-4367-bae1-f1886b909d5f None admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:53:49.308621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 660/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:53:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:53:59.117570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=30 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:53:59.304000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:54:53.029677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88a1d2-5f67-4e09-b5d5-903f2f11cde3 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:53.030221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88a1d2-5f67-4e09-b5d5-903f2f11cde3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:53.030221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88a1d2-5f67-4e09-b5d5-903f2f11cde3 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:53.031643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 647/1307] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:54:53.036607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088ca4-2f4c-40fb-b9e7-14f3840612b1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:53.037021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088ca4-2f4c-40fb-b9e7-14f3840612b1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:53.037234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088ca4-2f4c-40fb-b9e7-14f3840612b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:53.187866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-76088ca4-2f4c-40fb-b9e7-14f3840612b1 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:54:53.191301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-76088ca4-2f4c-40fb-b9e7-14f3840612b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:53.360745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 661/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:54:53.648245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:53.648528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:53.648926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:53.650587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 648/1309] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 17 19:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:54:53.656936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20654-dccf-45aa-a61f-cdc029e838fa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:53.657181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20654-dccf-45aa-a61f-cdc029e838fa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:53.657489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b20654-dccf-45aa-a61f-cdc029e838fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:53.744250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-76b20654-dccf-45aa-a61f-cdc029e838fa None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:54:53.745442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-76b20654-dccf-45aa-a61f-cdc029e838fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:53.767811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:54:53.840073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 662/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 17 19:54:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:54:53.849671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:54:53.851591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:53.983024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dbe83-982b-4cdd-a6db-6fabad998040 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:54:54.062442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:54:54.063144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:54.063459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:54.063791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:54.065452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:54.123393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:54:54.186766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:54:54.188379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096c1fc-6373-44b3-83c8-fd83bf6d2305 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:54:54.211415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 649/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:54:56.367669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:56.368250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:56.368250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:56.369608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 663/1312] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:54:56.374997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa83b-0352-4519-bf2d-3d1107cfd53f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:56.375405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa83b-0352-4519-bf2d-3d1107cfd53f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:56.375614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fa83b-0352-4519-bf2d-3d1107cfd53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:56.462477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-828fa83b-0352-4519-bf2d-3d1107cfd53f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:54:56.463779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-828fa83b-0352-4519-bf2d-3d1107cfd53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:56.570396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 650/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:54:56.852425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:54:56.854032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:56.975554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbbe72-d077-42ac-9bbf-28b86d642c4c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:54:57.050114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:54:57.052755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:54:57.053530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:57.053869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:57.054219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:57.057691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:57.172014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:54:57.173461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c781b-a2d7-4e9f-aa95-40badcdc8d78 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:54:57.197799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 664/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:54:59.437830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:59.438715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:59.438715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:59.440880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 651/1315] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:54:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:54:59.447132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a6e05-851c-4a3b-a1f1-22cd77befd82 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:54:59.447896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a6e05-851c-4a3b-a1f1-22cd77befd82 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:54:59.448025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a6e05-851c-4a3b-a1f1-22cd77befd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:54:59.521121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aa1a6e05-851c-4a3b-a1f1-22cd77befd82 None None] MFA Rules not processed for user `373336f834404161825914878c086dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:54:59.522303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aa1a6e05-851c-4a3b-a1f1-22cd77befd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:59.577988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:54:59.623448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 665/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:54:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:54:59.911865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:54:59.914490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:54:59.931203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:00.031446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150787b-8344-4f28-9047-73abb03728a7 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:00.122487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:00.123106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:00.123387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:00.123656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:00.125272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:00.188469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:00.245115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:00.246601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a29fa-8d7c-4a4d-85a2-5e38edf29b04 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:00.270061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 652/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:54:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:02.695297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:02.696161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:02.696161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:02.698330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 666/1318] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:02.703738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7002ec-b41d-4813-9aa1-25bed73f19c8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:02.704185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7002ec-b41d-4813-9aa1-25bed73f19c8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:02.704289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7002ec-b41d-4813-9aa1-25bed73f19c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:02.772141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bf7002ec-b41d-4813-9aa1-25bed73f19c8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:02.773342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bf7002ec-b41d-4813-9aa1-25bed73f19c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:02.890420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 653/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:03.175560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:03.178106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:03.291588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78cd7e-4ee9-4664-852b-5656d52bb8d1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:03.378753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:03.379356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:03.379635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:03.379930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:03.381562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:03.389178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:03.496068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:03.497528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b104f6-41ee-4bbb-ae95-352909a4dbe6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:03.522144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 667/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:06.106415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:06.107041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:06.107041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:06.108318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 654/1321] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:06.113636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a320203-fa39-4a33-a4c0-ff4d9dcf4dfe None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:06.114099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a320203-fa39-4a33-a4c0-ff4d9dcf4dfe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:06.114307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a320203-fa39-4a33-a4c0-ff4d9dcf4dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:06.184313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4a320203-fa39-4a33-a4c0-ff4d9dcf4dfe None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:06.185510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a320203-fa39-4a33-a4c0-ff4d9dcf4dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:06.245848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:06.286603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 668/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:06.560816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:06.562488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:06.583083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:06.670714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a34f2-fea9-4412-9ac5-201c4809a73c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:06.748421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:06.748916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:06.749192 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:06.749461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:06.751096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:06.823663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:06.862849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:06.864270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375fb5b-a302-46bb-84e6-aca760c77c9c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:06.884568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 655/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:08.820508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda45b-52b6-4aa6-81bf-d883553872ec None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:08.821057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda45b-52b6-4aa6-81bf-d883553872ec None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:08.821057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eda45b-52b6-4aa6-81bf-d883553872ec None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:08.822298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 669/1324] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:08.827685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587823-b75e-411b-af1a-635bdea6eef5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:08.828109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587823-b75e-411b-af1a-635bdea6eef5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:08.828228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a587823-b75e-411b-af1a-635bdea6eef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:08.893528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4a587823-b75e-411b-af1a-635bdea6eef5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:08.894760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4a587823-b75e-411b-af1a-635bdea6eef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:08.992515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 656/1325] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:09.281823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:09.282176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:09.282508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:09.283897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 670/1326] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 17 19:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:09.290402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3825-a09d-4b79-a1c2-1c4baadcbbfd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:09.290668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3825-a09d-4b79-a1c2-1c4baadcbbfd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:09.290958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3825-a09d-4b79-a1c2-1c4baadcbbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:09.323973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:09.371911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bd3e3825-a09d-4b79-a1c2-1c4baadcbbfd None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:09.373060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bd3e3825-a09d-4b79-a1c2-1c4baadcbbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:09.455612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 657/1327] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 17 19:55:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:09.464715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:09.466988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:09.580360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fac821-fdfb-4653-ab96-f5e61392d455 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:09.657300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:09.657862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:09.658125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:09.658388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:09.659987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:09.680085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:09.776392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:09.777950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3ee1-3fba-452b-86fa-53d1e2210b97 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:09.796745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 671/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:11.950705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:11.951217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:11.951217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:11.952608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 658/1329] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:11.957889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331e653-08dd-49c0-a11d-b41002e0ee57 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:11.958285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331e653-08dd-49c0-a11d-b41002e0ee57 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:11.958431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331e653-08dd-49c0-a11d-b41002e0ee57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:12.036042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9331e653-08dd-49c0-a11d-b41002e0ee57 None None] MFA Rules not processed for user `373336f834404161825914878c086dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:12.037194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9331e653-08dd-49c0-a11d-b41002e0ee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:12.106213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:12.139578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 672/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:12.421812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:12.423359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:12.535172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2558bcb-e647-4b6c-8ffb-275681b41be2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:12.563262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:12.627975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:12.628596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:12.628872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:12.629148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:12.630805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:12.753905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:12.755620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af29a19-5d00-4da5-b00d-6c81b8de5459 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:12.775719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 659/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:15.089203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:15.090144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:15.090144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:15.091741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 673/1332] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:15.097315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3e5d-0eb1-407b-adc6-50454f85fa30 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:15.097790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3e5d-0eb1-407b-adc6-50454f85fa30 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:15.097903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3e5d-0eb1-407b-adc6-50454f85fa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:15.165771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:15.169230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ab6c3e5d-0eb1-407b-adc6-50454f85fa30 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:15.171945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ab6c3e5d-0eb1-407b-adc6-50454f85fa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:15.286250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 660/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:15.566553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:15.568101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:15.685108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35533a-e526-43c4-a34c-110eb206f088 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:15.763245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:15.763852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:15.764121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:15.764387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:15.766013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:15.793474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:15.888395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:15.889885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749ea0-92fb-4b4c-b3a9-443cdf08c4b2 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:15.911771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 674/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:17.839652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:17.840202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:17.840202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:17.841558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 661/1335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:17.847698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585afd-b6a1-4773-ab4b-565e45c3bde7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:17.848471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585afd-b6a1-4773-ab4b-565e45c3bde7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:17.848471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a585afd-b6a1-4773-ab4b-565e45c3bde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:17.926662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a585afd-b6a1-4773-ab4b-565e45c3bde7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:17.927864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a585afd-b6a1-4773-ab4b-565e45c3bde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:18.008395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:18.032967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 675/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:18.320280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:18.321858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:18.429276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b4c27-6b7b-42ff-bb3e-7091e42bcf38 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:18.457594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:18.508076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:18.508688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:18.508964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:18.509236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:18.510873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:18.630114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:18.631560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3015fa-a4ce-4f59-a575-1a254167fbcc service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:18.649921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 662/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:21.030926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:21.031524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:21.031524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:21.032791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 676/1338] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:21.039292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353db9-0363-4cc7-98c5-45b0ea5c4341 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:21.040047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353db9-0363-4cc7-98c5-45b0ea5c4341 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:21.040346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0353db9-0363-4cc7-98c5-45b0ea5c4341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:21.115935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f0353db9-0363-4cc7-98c5-45b0ea5c4341 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:21.117156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f0353db9-0363-4cc7-98c5-45b0ea5c4341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:21.125997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:21.230899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 663/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:21.514845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:21.516441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:21.633041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830f558-eca0-4e53-9069-50d4793985d1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:21.712196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:21.712811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:21.713106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:21.713374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:21.715013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:21.750868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:21.826545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:21.827917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d14650-a867-4ff8-af1b-f0cb45bb3ffc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:21.846131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 677/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:25.761262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:25.761760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:25.761947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:25.763341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 664/1341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:25.768828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd314b-326e-489c-97f5-b31c5e431528 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:25.769290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd314b-326e-489c-97f5-b31c5e431528 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:25.769383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd314b-326e-489c-97f5-b31c5e431528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:25.837444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0acd314b-326e-489c-97f5-b31c5e431528 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:25.838722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0acd314b-326e-489c-97f5-b31c5e431528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:25.924643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:25.936844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 678/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:26.221302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:26.222926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:26.330398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793712d-53df-4985-8c61-93aee8eabb26 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:26.371168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:26.408393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:26.409001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:26.409275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:26.409552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:26.411178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:26.523325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:26.524660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5350f2-d463-447e-929b-224fde9ce587 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:26.543171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 665/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:29.033378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:29.033991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:29.033991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:29.035385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 679/1344] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:29.041298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cfda-3467-4bf2-b778-06c624da5a64 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:29.041873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cfda-3467-4bf2-b778-06c624da5a64 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:29.041873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09cfda-3467-4bf2-b778-06c624da5a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:29.115601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c09cfda-3467-4bf2-b778-06c624da5a64 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:29.116835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c09cfda-3467-4bf2-b778-06c624da5a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:29.136725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:29.218135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 666/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:29.498213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:29.499820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:29.614869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab671f3d-bae7-4a5b-a3d1-8ac37a840259 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:29.692056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:29.692729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:29.693009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:29.693267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:29.695074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:29.751349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:29.818491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:29.819946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb0f10-d326-4a91-9ae4-7667a8e4e2b1 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:29.838579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 680/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:34.011962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:34.012528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:34.012528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:34.013946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 667/1347] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:34.019783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc971-9cf9-4005-90ce-393a5969172b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:34.020262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc971-9cf9-4005-90ce-393a5969172b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:34.020262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbc971-9cf9-4005-90ce-393a5969172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:34.094008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d3fbc971-9cf9-4005-90ce-393a5969172b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:34.095266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d3fbc971-9cf9-4005-90ce-393a5969172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:34.195761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:34.198672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 681/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:34.486506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:34.488875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:34.608843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4c14e-d953-424f-8827-884138a3a266 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:34.657577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:34.684335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:34.684956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:34.685242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:34.685519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:34.687210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:34.803595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:34.805354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dcae52-49e5-4646-b1cc-dd0ebd7bc5c9 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:34.825333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 668/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:39.696824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502670-12f3-4e4a-b425-45fc68b28cf4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:39.697372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502670-12f3-4e4a-b425-45fc68b28cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:39.697372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46502670-12f3-4e4a-b425-45fc68b28cf4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:39.701682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 682/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 19:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:39.706242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb854af4-dcba-49ec-bbe4-08afb932ed6d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:39.707070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb854af4-dcba-49ec-bbe4-08afb932ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:39.707551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb854af4-dcba-49ec-bbe4-08afb932ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:39.816575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bb854af4-dcba-49ec-bbe4-08afb932ed6d None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:39.818315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bb854af4-dcba-49ec-bbe4-08afb932ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:39.856983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:39.995877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 669/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 19:55:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:44.001184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5cc1-d108-4024-ab54-66af9da357c4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:44.001846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5cc1-d108-4024-ab54-66af9da357c4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:44.001846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5cc1-d108-4024-ab54-66af9da357c4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:44.003322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 683/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:44.009857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db590e-bff7-4964-91f7-2cd8f50bc8dc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:44.010283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db590e-bff7-4964-91f7-2cd8f50bc8dc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:44.010386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db590e-bff7-4964-91f7-2cd8f50bc8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:44.093636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f5db590e-bff7-4964-91f7-2cd8f50bc8dc None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:44.094868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5db590e-bff7-4964-91f7-2cd8f50bc8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:44.176397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:44.216167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 670/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 19:55:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:45.531596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede775-fccb-4049-aade-df68394cbd31 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:45.532141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede775-fccb-4049-aade-df68394cbd31 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:45.532141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ede775-fccb-4049-aade-df68394cbd31 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:45.533431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 684/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 19:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:45.538960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888c8c-7d49-4302-a4df-c324df02eb36 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:45.539429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888c8c-7d49-4302-a4df-c324df02eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:45.540031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888c8c-7d49-4302-a4df-c324df02eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:45.618174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4f888c8c-7d49-4302-a4df-c324df02eb36 None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:45.619387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4f888c8c-7d49-4302-a4df-c324df02eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:45.733585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 671/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 19:55:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:49.138153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41c1d-a288-4b25-8c0a-b2f22acad04c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:49.138720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41c1d-a288-4b25-8c0a-b2f22acad04c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:49.138720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e41c1d-a288-4b25-8c0a-b2f22acad04c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:49.139968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 685/1356] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:49.145806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0c8d0-6474-4012-8826-a20b0b9a2ad4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:49.146217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0c8d0-6474-4012-8826-a20b0b9a2ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:49.146402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0c8d0-6474-4012-8826-a20b0b9a2ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:49.198862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:49.236532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-afb0c8d0-6474-4012-8826-a20b0b9a2ad4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:49.238975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-afb0c8d0-6474-4012-8826-a20b0b9a2ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:49.365441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 672/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:49.662874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:49.663490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:49.663936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:49.666168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 686/1358] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 17 19:55:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:49.675123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef2ab-4afa-402f-978c-baf8f4e6a539 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:49.675868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef2ab-4afa-402f-978c-baf8f4e6a539 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:49.676565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef2ab-4afa-402f-978c-baf8f4e6a539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:49.760531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-af4ef2ab-4afa-402f-978c-baf8f4e6a539 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:49.761819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-af4ef2ab-4afa-402f-978c-baf8f4e6a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:49.836231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:49.921113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 673/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 17 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:49.930834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:49.932375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:50.087356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ebbfc-6244-4595-a816-21e2d21935b2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:50.178857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:50.221413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:50.223153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:50.223578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:50.224008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:50.225920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:50.318513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:50.320711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b9719-df1b-4eee-ae8e-b785f03d4f2a service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:50.342232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 687/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:55:52.426376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:52.426960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:52.426960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:52.428506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 674/1361] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:52.436738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e26e18-0e8a-4130-835b-045b46348adc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:52.437297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e26e18-0e8a-4130-835b-045b46348adc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:52.437771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e26e18-0e8a-4130-835b-045b46348adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:52.507516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20e26e18-0e8a-4130-835b-045b46348adc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:52.508714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20e26e18-0e8a-4130-835b-045b46348adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:52.555447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:52.629794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 688/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:52.904852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:52.906383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:53.057786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78f3a-0195-4a95-802a-b1a1423ba085 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:53.068186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:53.180049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:53.180743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:53.181081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:53.181803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:53.184146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:53.283703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:53.285368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa7c51-7764-4ca7-8d66-4f8666ade801 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:53.307700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 675/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:55.535581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db42c8-62b4-4d7d-a494-6099189f6d1c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:55.536145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db42c8-62b4-4d7d-a494-6099189f6d1c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:55.536145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db42c8-62b4-4d7d-a494-6099189f6d1c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:55.537450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 689/1364] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:55.543151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c035f-0258-486b-8713-0c96e9e5d91a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:55.543938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c035f-0258-486b-8713-0c96e9e5d91a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:55.544379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c035f-0258-486b-8713-0c96e9e5d91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:55.562242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:55.619727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-dc2c035f-0258-486b-8713-0c96e9e5d91a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:55.621266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dc2c035f-0258-486b-8713-0c96e9e5d91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:55.729045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 676/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:56.017398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:56.017773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:56.017985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:56.019322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 690/1366] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 17 19:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:56.024839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff109c6-8bda-4336-bec3-a5e3ed397cb9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:56.025196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff109c6-8bda-4336-bec3-a5e3ed397cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:56.025608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff109c6-8bda-4336-bec3-a5e3ed397cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:56.107849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2ff109c6-8bda-4336-bec3-a5e3ed397cb9 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:56.109107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2ff109c6-8bda-4336-bec3-a5e3ed397cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:56.147038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:56.235897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 677/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 17 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:56.243806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:56.245600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:56.406648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60f95f-9e84-4347-b912-4ccd4083f3a6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:56.429425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:56.527385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:56.528069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:56.528443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:56.528814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:56.530595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:56.643134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:56.644931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aaa82-0cf4-4457-bec3-7c988a7f35a3 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:56.664065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 691/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:57.179554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c9d6-fc9a-4f6e-af56-ce542a7db415 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:57.180125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c9d6-fc9a-4f6e-af56-ce542a7db415 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:57.180125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c9d6-fc9a-4f6e-af56-ce542a7db415 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:57.181667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 678/1369] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:57.187070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c41ad-bf8b-412a-9634-881eee61ace2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:57.187315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c41ad-bf8b-412a-9634-881eee61ace2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:57.187613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c41ad-bf8b-412a-9634-881eee61ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:57.222882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:57.260141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-180c41ad-bf8b-412a-9634-881eee61ace2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:57.261446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-180c41ad-bf8b-412a-9634-881eee61ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:57.362538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 692/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:57.905450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:57.905798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:57.906031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:57.907587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 679/1371] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 17 19:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:57.912412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263ad51-95d7-443f-b3a0-853072614849 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:57.912893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263ad51-95d7-443f-b3a0-853072614849 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:57.913276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263ad51-95d7-443f-b3a0-853072614849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:57.991121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6263ad51-95d7-443f-b3a0-853072614849 None None] MFA Rules not processed for user `c97be7f3685b44cfa26aeae0f5b5d931`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:57.992490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6263ad51-95d7-443f-b3a0-853072614849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:58.041797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:58.076113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 693/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 17 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:58.083950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:58.085846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:58.196216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f462a-ace6-48dd-b9cb-2d63c08370e4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:58.238147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:58.272490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:58.274211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:58.274505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:58.274784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:58.276531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:58.400410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:58.402226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d02b18-35fc-4f7c-b108-fae2223e93a6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:58.422183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 680/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:55:58.632208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:58.632464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:58.632635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:58.634154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 694/1374] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:55:58.639580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccff1d1-44f6-4fa6-aded-c7940d8bffff None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:58.639988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccff1d1-44f6-4fa6-aded-c7940d8bffff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:58.640178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccff1d1-44f6-4fa6-aded-c7940d8bffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:58.728516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5ccff1d1-44f6-4fa6-aded-c7940d8bffff None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:55:58.729926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5ccff1d1-44f6-4fa6-aded-c7940d8bffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:58.758604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:58.837002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 681/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:55:59.125213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:55:59.126778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:59.281813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e222b72-1d63-4cf3-b0f6-5af07bec6a34 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:55:59.333167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:55:59.396825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:55:59.397431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:55:59.397741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:55:59.398011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:55:59.399674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:55:59.490199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:55:59.491812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4477-6e2c-410c-b769-18f2846ad65d service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:55:59.510388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 695/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:01.443615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:01.444084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:01.444179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:01.445523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 682/1377] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:01.450896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c9ad-2229-4eda-a04d-fb37ada4e8d5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:01.451334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c9ad-2229-4eda-a04d-fb37ada4e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:01.451552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c9ad-2229-4eda-a04d-fb37ada4e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:01.527530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5920c9ad-2229-4eda-a04d-fb37ada4e8d5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:01.528829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5920c9ad-2229-4eda-a04d-fb37ada4e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:01.537318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:01.634661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 696/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:01.928742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:01.930169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:02.129069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9669d9-89ab-40dd-a22d-a123559add94 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:02.133362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:02.248288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:02.248897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:02.249167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:02.249430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:02.251036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:02.345184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:02.346833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af09df-7aa6-4fb4-9bcc-8814bbb54d1b service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:02.365395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 683/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:04.303018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:04.303537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:04.303537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:04.305202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 697/1380] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:04.310853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0f46-1aaf-425c-907f-69cfcef3a26f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:04.311362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0f46-1aaf-425c-907f-69cfcef3a26f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:04.311362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec0f46-1aaf-425c-907f-69cfcef3a26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:04.323108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:04.381574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-26ec0f46-1aaf-425c-907f-69cfcef3a26f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:04.382813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-26ec0f46-1aaf-425c-907f-69cfcef3a26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:04.484506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 684/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:04.777023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:04.778805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:04.922610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:04.935010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4bfdb5-7956-4d92-a3da-ed9388fd4578 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:05.061010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:05.061722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:05.061999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:05.062261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:05.063857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:05.159435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:05.161052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bd84b-3bbe-4cb5-bf1e-0f2a3ae2c7c9 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:05.171686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:05.181640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 698/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:56:07.122426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:07.123073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:07.123185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:07.124890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 685/1383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:07.131351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038abb-4b9e-48e8-ab42-7f87551e9e48 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:07.131828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038abb-4b9e-48e8-ab42-7f87551e9e48 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:07.132046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8038abb-4b9e-48e8-ab42-7f87551e9e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:07.200658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c8038abb-4b9e-48e8-ab42-7f87551e9e48 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:07.201943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8038abb-4b9e-48e8-ab42-7f87551e9e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:07.341053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 699/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:07.774275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:07.776023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:07.864881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:07.934852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7e992-910d-4815-b79c-a819cddf1b78 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:08.057309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:08.057961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:08.058266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:08.058537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:08.060163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:08.111340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:08.155628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:08.157953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be96e4-4190-47d4-bd54-3ad5bc372197 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:08.176249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 686/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:10.526901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:10.527521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:10.527521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:10.529270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 700/1386] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:10.535616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f2f0-88bc-46bd-bc98-9467edde3e18 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:10.536170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f2f0-88bc-46bd-bc98-9467edde3e18 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:10.536170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376f2f0-88bc-46bd-bc98-9467edde3e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:10.613877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7376f2f0-88bc-46bd-bc98-9467edde3e18 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:10.615443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7376f2f0-88bc-46bd-bc98-9467edde3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:10.727487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 687/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:11.081400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:11.083352 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:11.169173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:11.244939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e691f5-769f-4b7c-8cea-4a43ce2f3753 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:11.365471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:11.366311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:11.366650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:11.366975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:11.368983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:11.416115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:11.470113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:11.471769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee111492-1e85-4d27-9124-c1749424140e service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:11.490176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 701/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:13.413554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6dbe-3e65-44f1-9e77-1234c47a272e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:13.414169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6dbe-3e65-44f1-9e77-1234c47a272e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:13.414169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e6dbe-3e65-44f1-9e77-1234c47a272e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:13.415443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 688/1389] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:13.421024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c194e-8dc0-4c3f-9d15-782d7d4e47d9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:13.421461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c194e-8dc0-4c3f-9d15-782d7d4e47d9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:13.421582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c194e-8dc0-4c3f-9d15-782d7d4e47d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:13.490158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-444c194e-8dc0-4c3f-9d15-782d7d4e47d9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:13.491366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-444c194e-8dc0-4c3f-9d15-782d7d4e47d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:13.599821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 702/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:13.653141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:13.653382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:13.653722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:13.655288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 689/1391] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:13.660302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476cfc8-c614-44c3-95a7-aaef1acbcffc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:13.660579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476cfc8-c614-44c3-95a7-aaef1acbcffc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:13.660846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476cfc8-c614-44c3-95a7-aaef1acbcffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:13.678222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:13.729755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e476cfc8-c614-44c3-95a7-aaef1acbcffc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:13.730980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e476cfc8-c614-44c3-95a7-aaef1acbcffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:13.828763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 703/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:13.889927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:13.891543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:13.924664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:14.040575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc96161-e6b7-4929-909a-eecca298c254 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:14.124235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c894-ae43-4d8f-915c-c12dba5f15b5 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:14.124507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c894-ae43-4d8f-915c-c12dba5f15b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:14.124800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8c894-ae43-4d8f-915c-c12dba5f15b5 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:14.126183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 704/1393] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 17 19:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:56:14.130801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477739aa-bb7d-43e4-b087-4a3284c11985 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:14.131044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477739aa-bb7d-43e4-b087-4a3284c11985 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:14.131315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477739aa-bb7d-43e4-b087-4a3284c11985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:14.146468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:14.160524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:14.161399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:14.161711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:14.162096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:14.163779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:14.208725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-477739aa-bb7d-43e4-b087-4a3284c11985 None None] MFA Rules not processed for user `9f03a7d5287c48e4821d308767f4c141`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:14.209978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-477739aa-bb7d-43e4-b087-4a3284c11985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:14.245205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:14.254039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:14.255704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:14.274018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 690/1394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:14.289745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 705/1395] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 17 19:56:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:56:14.297917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:14.299087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:14.379189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b55e7-d969-4524-b00f-4a49af9ee086 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:14.426110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:14.462804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:14.463443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:14.463739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:14.463980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:14.465760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:14.580622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:14.582109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135645-dd33-4f7d-afb2-09149428f480 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:14.600652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 691/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:15.337817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654cc21-e382-46d6-b852-c5026a9fdc78 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:15.338305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654cc21-e382-46d6-b852-c5026a9fdc78 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:15.338415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654cc21-e382-46d6-b852-c5026a9fdc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:15.409869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7654cc21-e382-46d6-b852-c5026a9fdc78 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:15.411084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7654cc21-e382-46d6-b852-c5026a9fdc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:15.498005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 706/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:15.543107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:15.543634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:15.543634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:15.545156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 692/1398] 10.222.0.18 () {54 vars in 943 bytes} [Wed Aug 17 19:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:15.551826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418f70-1dfc-426d-a922-4486039766c3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:15.552203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418f70-1dfc-426d-a922-4486039766c3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:15.552506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418f70-1dfc-426d-a922-4486039766c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:15.615763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:15.640339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b8418f70-1dfc-426d-a922-4486039766c3 None None] MFA Rules not processed for user `103c6c2fb3104914b8b4ec7c196f4996`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:15.641974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b8418f70-1dfc-426d-a922-4486039766c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:15.754205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 707/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Aug 17 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:15.763965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:15.765562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:15.845988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:15.907840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181bd5f5-0e83-40a8-8518-6e16a8430482 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:16.019382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:16.020005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:16.020265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:16.020548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:16.022223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:16.082405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:16.107959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:16.109584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5836721-25fe-4a9a-8e53-753b0d73f344 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:16.117842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 693/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 19:56:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 19:56:16.218554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:16.218926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:16.219203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:16.220902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 708/1401] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:16.226659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027701-f130-4875-ae09-f39710f9c174 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:16.226952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027701-f130-4875-ae09-f39710f9c174 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:16.227227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027701-f130-4875-ae09-f39710f9c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:16.307978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1c027701-f130-4875-ae09-f39710f9c174 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:16.310729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1c027701-f130-4875-ae09-f39710f9c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:16.415892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 694/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:16.706106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:16.708071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:16.828331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:16.873901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53ed9d-cea6-46be-a452-54c5a75582ae None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:16.997191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:16.997802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:16.998081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:16.998351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:17.001866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:17.079122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:17.102234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:17.103889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1fbe6b-79aa-41c5-8965-d12a82cd65c8 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:17.123326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 709/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:19.046393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:19.046990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:19.046990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:19.048785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 695/1404] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:19.053992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ae804-8f78-4bb9-b037-080ae8f20bb2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:19.054421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ae804-8f78-4bb9-b037-080ae8f20bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:19.054509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ae804-8f78-4bb9-b037-080ae8f20bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:19.126298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eb7ae804-8f78-4bb9-b037-080ae8f20bb2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:19.127528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eb7ae804-8f78-4bb9-b037-080ae8f20bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:19.293657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 710/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:19.580790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:19.582482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:19.642620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:19.803962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43968908-64f2-4879-83e4-9ae5d1b4b388 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:19.947970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:19.948695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:19.949571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:19.949571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:19.951352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:19.966811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:20.076902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:20.079933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae803472-e92e-42b5-8284-323344f7e2a8 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:20.104290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 696/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:22.039285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:22.039893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:22.039893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:22.041298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 711/1407] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:22.046755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75b8f3-b3a5-497b-9b38-8ec37a2f56c0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:22.048108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75b8f3-b3a5-497b-9b38-8ec37a2f56c0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:22.048880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75b8f3-b3a5-497b-9b38-8ec37a2f56c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:22.135908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0e75b8f3-b3a5-497b-9b38-8ec37a2f56c0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:22.137255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e75b8f3-b3a5-497b-9b38-8ec37a2f56c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:22.232128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:22.253723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 697/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:22.529930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:22.533491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:22.604906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:22.709064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295527c-259f-4f0d-8483-f44f1165111c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:22.832762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:22.833426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:22.833799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:22.834076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:22.839504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:22.850525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:22.940591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:22.942255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d3ca8-893b-4896-81a9-29bf0f315124 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:22.961435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 712/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:32.244065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:32.959233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=41 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:37.801501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acb7ec-0297-42ea-8945-b6f9418b791a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:37.802409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acb7ec-0297-42ea-8945-b6f9418b791a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:37.802409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acb7ec-0297-42ea-8945-b6f9418b791a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:37.804644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 698/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 17 19:56:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:37.812915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c0f0-4ccb-4b62-b724-227023d14673 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:37.813593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c0f0-4ccb-4b62-b724-227023d14673 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:37.813703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369c0f0-4ccb-4b62-b724-227023d14673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:37.899512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8369c0f0-4ccb-4b62-b724-227023d14673 None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:37.900736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8369c0f0-4ccb-4b62-b724-227023d14673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:38.010675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 713/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 17 19:56:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:38.025412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef9a7-1a7f-4a47-a29c-b311522e10e6 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:38.025756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef9a7-1a7f-4a47-a29c-b311522e10e6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:38.025963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eef9a7-1a7f-4a47-a29c-b311522e10e6 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:38.027381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 699/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 17 19:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:38.033809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cde9-e48e-47db-bb10-a982b4be987b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:38.034099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cde9-e48e-47db-bb10-a982b4be987b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:38.034370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cde9-e48e-47db-bb10-a982b4be987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:38.108121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:38.111426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d775cde9-e48e-47db-bb10-a982b4be987b None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:38.113139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d775cde9-e48e-47db-bb10-a982b4be987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:38.236965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 714/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 17 19:56:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:56:40.607871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:40.608347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:40.608451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:40.609796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:40.615047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5173-617e-4933-8f61-26d5164ef3f7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:40.615516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5173-617e-4933-8f61-26d5164ef3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:40.615516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f5173-617e-4933-8f61-26d5164ef3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:40.698110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-de6f5173-617e-4933-8f61-26d5164ef3f7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:40.699387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-de6f5173-617e-4933-8f61-26d5164ef3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:40.756584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:40.799813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:41.088753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:41.090450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:41.251556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8725386-ad3c-47e2-af99-c150b7d3a2f5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:41.336525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:41.387030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:41.387652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:41.387932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:41.388200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:41.389856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:41.483941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:41.485591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c425222-fdb5-4b5b-9e4c-ca579203f32f service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:41.505246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 701/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:42.406635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:42.407217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:42.407217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:42.408525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:42.415141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ef698-d300-4c75-9994-9d49d9524eff None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:42.415828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ef698-d300-4c75-9994-9d49d9524eff None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:42.416072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ef698-d300-4c75-9994-9d49d9524eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:42.496760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a22ef698-d300-4c75-9994-9d49d9524eff None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:42.497995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a22ef698-d300-4c75-9994-9d49d9524eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:42.526225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:42.602130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:42.753665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:42.756000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:42.927502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e8a72-3841-4af7-a261-aa5dd2da5784 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:42.968162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:43.049385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:43.049919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:43.050190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:43.050453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:43.052050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:43.142995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:43.144612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7294432-1b6f-48f5-98a2-45d4c4711cd4 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:43.162862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 717/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:44.032019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:44.032972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:44.032972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:44.035244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:44.040639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e16-8178-475c-8e54-39454e698953 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:44.041093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e16-8178-475c-8e54-39454e698953 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:44.041254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60545e16-8178-475c-8e54-39454e698953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:44.099842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:44.125145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-60545e16-8178-475c-8e54-39454e698953 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:44.126490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-60545e16-8178-475c-8e54-39454e698953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:44.233709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:44.388908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:44.391463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:44.549897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2094170-695f-43d9-a8ad-98fabb89666b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:44.562413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:44.669821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:44.670360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:44.670625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:44.670901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:44.672836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:44.769919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:44.771546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf451c-14c6-4afb-997d-4909c3221f47 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:44.789438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 704/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:45.649150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:45.649979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:45.649979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:45.651635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:45.657484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d476-326a-4423-84eb-b91a9873aa18 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:45.657936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d476-326a-4423-84eb-b91a9873aa18 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:45.658043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2d476-326a-4423-84eb-b91a9873aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:45.686701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:45.745562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3bb2d476-326a-4423-84eb-b91a9873aa18 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:45.746709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bb2d476-326a-4423-84eb-b91a9873aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:45.857735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:46.011402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:46.012885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:46.143732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:46.181169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8a63d-8f6b-495f-8652-c03a331b316c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:46.304285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:46.304801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:46.305172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:46.305446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:46.307181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:46.387382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:46.401512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:46.403180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89088f65-11b9-41de-b21d-88caa4d9bac1 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:46.421566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 720/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:47.291895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:47.292527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:47.292527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:47.293926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:56:47.300581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38335033-65d8-4da3-8511-a0d2798f8067 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:47.301287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38335033-65d8-4da3-8511-a0d2798f8067 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:47.301699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38335033-65d8-4da3-8511-a0d2798f8067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:47.384843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-38335033-65d8-4da3-8511-a0d2798f8067 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:47.386094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-38335033-65d8-4da3-8511-a0d2798f8067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:47.486925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:47.637899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:47.639461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:47.726768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:47.789957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dae709-4cc6-437c-8869-d6a420c282f8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:47.914095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:47.914705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:47.914985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:47.915373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:47.916963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:47.969820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:48.011009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:48.012631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392d239-005b-4a7d-bc50-b6240a8b4822 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:48.036803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 707/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:48.837861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:48.838436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:48.838436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:48.839766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:48.845933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f04b1-6fb7-4f02-9a39-f0fe75f39fdb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:48.846619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f04b1-6fb7-4f02-9a39-f0fe75f39fdb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:48.846781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f04b1-6fb7-4f02-9a39-f0fe75f39fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:48.938216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f25f04b1-6fb7-4f02-9a39-f0fe75f39fdb None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:48.940018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f25f04b1-6fb7-4f02-9a39-f0fe75f39fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:49.069880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:49.222405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:49.223926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:49.278841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:49.390623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedac771-6c55-40a8-9c5b-f46c92e70d8f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:49.506155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:49.506676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:49.506953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:49.507217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:49.508801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:49.516195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:49.610662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:49.612285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0ebb3-ff3a-40e4-96f3-1472c1e884cd service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:49.632992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 723/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:56:50.430341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:50.430955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:50.430955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:50.432361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:50.437462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148c6f-c1da-458b-bd33-b6594d6f150b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:50.437917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148c6f-c1da-458b-bd33-b6594d6f150b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:50.438140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148c6f-c1da-458b-bd33-b6594d6f150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:50.517780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d4148c6f-c1da-458b-bd33-b6594d6f150b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:50.519004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4148c6f-c1da-458b-bd33-b6594d6f150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:50.594561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:50.625513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:50.774962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:50.776440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:50.794335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:50.929672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5694389-6afa-4b31-853e-3c3c9ff48d57 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:51.027482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:51.064117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:51.064733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:51.065006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:51.065258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:51.066933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:51.164760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:51.166435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f896b4-b17f-4982-9c69-78694e6e559e service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:51.185284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 710/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:51.995492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:51.996317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:51.996317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:51.998286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:52.004349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97069bff-6dee-43d4-a7c4-1341501d0d95 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:52.004823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97069bff-6dee-43d4-a7c4-1341501d0d95 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:52.004924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97069bff-6dee-43d4-a7c4-1341501d0d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:52.092128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-97069bff-6dee-43d4-a7c4-1341501d0d95 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:52.093371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-97069bff-6dee-43d4-a7c4-1341501d0d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:52.134453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:52.192184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:52.346229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:52.348860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:52.510180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebec02-972c-43c0-8d96-794c5f045861 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:52.573000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:52.635068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:52.635686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:52.635955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:52.636218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:52.637866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:52.734396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:52.736055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7e6c9-f2b1-48fb-bd99-1061c9978979 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:52.755485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 726/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:53.554349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:53.555010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:53.555010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:53.556271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 19:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:53.561469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b044-3c46-41e7-b4db-d3cf094436ed None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:53.561964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b044-3c46-41e7-b4db-d3cf094436ed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:53.562077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5b044-3c46-41e7-b4db-d3cf094436ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:53.642176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34c5b044-3c46-41e7-b4db-d3cf094436ed None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:53.643397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34c5b044-3c46-41e7-b4db-d3cf094436ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:53.645482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:53.743416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 19:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:53.900081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:53.901846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:54.055810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104dcb4-da2d-442d-84e0-4818443883ed None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:54.095601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:54.188602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:54.189133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:54.189410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:54.189699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:54.191326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:54.285655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:54.287257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f262b36-d99a-4471-a261-46e1bc4f7317 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:54.306439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 713/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:56:56.105451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69509a4a-0078-477e-b206-53c497ab75db None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:56.106095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69509a4a-0078-477e-b206-53c497ab75db None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:56.106095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69509a4a-0078-477e-b206-53c497ab75db None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:56.107388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:56.113947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf9d49-6dd5-49d1-a1ed-9e8b43cbe1a0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:56.114686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf9d49-6dd5-49d1-a1ed-9e8b43cbe1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:56.114839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf9d49-6dd5-49d1-a1ed-9e8b43cbe1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:56.159544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:56.187616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2edf9d49-6dd5-49d1-a1ed-9e8b43cbe1a0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:56.188888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2edf9d49-6dd5-49d1-a1ed-9e8b43cbe1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:56.302494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:56.590398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69509a4a-0078-477e-b206-53c497ab75db None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:56.591952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-69509a4a-0078-477e-b206-53c497ab75db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:56.707134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69509a4a-0078-477e-b206-53c497ab75db None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:56.741662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:56.789499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:56.790150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:56.790426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:56.790689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:56.792311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:56.910931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:56.912392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b684c3-155a-4912-9665-19878539dc17 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:56.932215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 729/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 19:56:59.061751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:59.062469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:59.062469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:59.064663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:56:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:56:59.070507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cccb1-b942-461b-a0f8-ba756076c5f6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:59.070963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cccb1-b942-461b-a0f8-ba756076c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:59.071063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944cccb1-b942-461b-a0f8-ba756076c5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:59.156475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-944cccb1-b942-461b-a0f8-ba756076c5f6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:56:59.157778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-944cccb1-b942-461b-a0f8-ba756076c5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:59.171377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:59.271893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:56:59.558308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:56:59.559853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:59.664916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd16c2-244f-4059-be40-3e72322a3934 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:56:59.677764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:56:59.740501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:56:59.741140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:56:59.741418 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:56:59.741689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:56:59.743303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:56:59.854986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:56:59.856427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0d524-9914-4773-9f74-8e67e274cc90 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:56:59.876019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 716/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:02.069698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:02.070603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:02.070603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:02.072980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:02.078682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98afe-3f06-49bf-bd34-68fbe7628213 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:02.079137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98afe-3f06-49bf-bd34-68fbe7628213 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:02.079228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98afe-3f06-49bf-bd34-68fbe7628213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:02.127806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:02.147465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b4f98afe-3f06-49bf-bd34-68fbe7628213 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:02.148699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b4f98afe-3f06-49bf-bd34-68fbe7628213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:02.256119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:02.538545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:02.540131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:02.652588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6903a-04ee-48b0-b40d-27323e03ead4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:02.695733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:02.731259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:02.731887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:02.732160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:02.732423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:02.734054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:02.852225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:02.853738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826ae82-de5d-414b-88c6-3342c7c2ef8e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:02.873150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 732/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:04.881831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:04.882702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:04.882702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:04.884893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:04.892960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307041-fe0d-4722-9e09-c163b572e5a9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:04.893384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307041-fe0d-4722-9e09-c163b572e5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:04.893501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4307041-fe0d-4722-9e09-c163b572e5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:04.966674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f4307041-fe0d-4722-9e09-c163b572e5a9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:04.967885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4307041-fe0d-4722-9e09-c163b572e5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:04.989722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:05.072385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:05.359456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:05.361927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:05.483966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d087446-50d6-4521-8b8d-c86a09693872 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:05.511519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:05.574065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:05.574696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:05.574978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:05.575251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:05.576912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:05.693581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:05.694915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a90b9-5839-4e9b-9558-110949e8c930 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:05.713277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 719/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:07.672767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:07.673674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:07.673674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:07.675971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:07.684723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ff3ee-7bc8-4dd8-a2fb-f695320022d5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:07.685409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ff3ee-7bc8-4dd8-a2fb-f695320022d5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:07.685629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ff3ee-7bc8-4dd8-a2fb-f695320022d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:07.757861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:07.760898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e79ff3ee-7bc8-4dd8-a2fb-f695320022d5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:07.763014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e79ff3ee-7bc8-4dd8-a2fb-f695320022d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:07.864482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:08.148143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:08.149739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:08.265342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23aa8b-01b4-49b1-8359-c8d13def3dbd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:08.316478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:08.349794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:08.350394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:08.350664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:08.350927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:08.352714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:08.469440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:08.470878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ba829-e679-4d93-a517-9156c89f2a82 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:08.489633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 735/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:10.464564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:10.465145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:10.465145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:10.466535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:10.472026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fc2e7-a210-44a8-bc38-c0c4d3537044 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:10.472475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fc2e7-a210-44a8-bc38-c0c4d3537044 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:10.472570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fc2e7-a210-44a8-bc38-c0c4d3537044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:10.539850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e99fc2e7-a210-44a8-bc38-c0c4d3537044 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:10.541063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e99fc2e7-a210-44a8-bc38-c0c4d3537044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:10.572897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:10.645340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:10.928024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:10.930635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:11.043217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccd240-1ac5-4874-bc99-ecf67709b994 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:11.072072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:11.117894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:11.118643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:11.118910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:11.119231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:11.120919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:11.230491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:11.231841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f82d2-3cab-4e2e-b01b-921466380f2d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:11.250165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 722/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:13.254626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:13.255531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:13.255531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:13.257613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:13.263235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69ab0-9a41-4593-871b-e4442e45341d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:13.263669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69ab0-9a41-4593-871b-e4442e45341d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:13.263834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d69ab0-9a41-4593-871b-e4442e45341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:13.338594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a2d69ab0-9a41-4593-871b-e4442e45341d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:13.339818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a2d69ab0-9a41-4593-871b-e4442e45341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:13.348291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:13.446121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:13.733738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:13.735329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:13.865149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67816d67-b3c7-4709-a491-a22e7bb56e07 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:13.930402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:13.950064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:13.950666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:13.950944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:13.951208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:13.952821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:14.075121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:14.076593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad9343-8ff4-413f-8686-e471555085cb service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:14.096617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 738/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:16.036612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:16.037251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:16.037251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:16.038559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:16.043989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539bdc4-5f54-4aa7-a027-bd0f7b7470e2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:16.044413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539bdc4-5f54-4aa7-a027-bd0f7b7470e2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:16.044525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539bdc4-5f54-4aa7-a027-bd0f7b7470e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:16.113000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2539bdc4-5f54-4aa7-a027-bd0f7b7470e2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:16.114230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2539bdc4-5f54-4aa7-a027-bd0f7b7470e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:16.155270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:16.220915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:16.516564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:16.519064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:16.760623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5fa64-e840-44ba-884e-b92108aa4453 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:16.805173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:16.843095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:16.843710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:16.843984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:16.844242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:16.845866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:16.960982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:16.962451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14fbaf-979b-4456-832b-e2d64bad2427 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:16.980683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 725/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:18.906414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:18.907003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:18.907003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:18.908356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 19:57:18.914882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519b82d-60bb-41fd-b719-39e61c5aee51 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:18.915678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519b82d-60bb-41fd-b719-39e61c5aee51 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:18.915678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519b82d-60bb-41fd-b719-39e61c5aee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:18.985522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c519b82d-60bb-41fd-b719-39e61c5aee51 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:18.986810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c519b82d-60bb-41fd-b719-39e61c5aee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:19.008143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:19.097432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:19.376897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:19.378506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:19.490719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648cc9f-6e21-480a-b9ff-bcfd6946c326 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:19.561722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:19.569127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:19.569769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:19.570044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:19.570308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:19.571932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:19.689036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:19.690536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5a4e5-0522-4f46-9bd3-0a22b17f9ede service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:19.709279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 741/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:21.654596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:21.655483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:21.655483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:21.657596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 19:57:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:21.663214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16822602-bfed-4972-832b-a78420073be7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:21.663694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16822602-bfed-4972-832b-a78420073be7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:21.663694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16822602-bfed-4972-832b-a78420073be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:21.733164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-16822602-bfed-4972-832b-a78420073be7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:21.734421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-16822602-bfed-4972-832b-a78420073be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:21.795273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:21.848031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 19:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:22.126976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:22.128555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:22.242988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eca0a5-db81-44bf-ba32-ff7b2f25434a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:22.300363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:22.329395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:22.330036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:22.330310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:22.330576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:22.332202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:22.450064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:22.451508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8190a95-f681-43b3-886b-ac172a1868d9 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:22.469905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 728/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:31.846062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=22 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:32.467450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=66 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:41.718578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde590-071e-48ca-95eb-02e7ac54f2b6 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:41.719191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde590-071e-48ca-95eb-02e7ac54f2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:41.719191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde590-071e-48ca-95eb-02e7ac54f2b6 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:41.720457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 743/1471] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:41.726875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779bf3-3ca9-48c3-b6c7-ba148aff51c7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:41.727670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779bf3-3ca9-48c3-b6c7-ba148aff51c7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:41.727834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779bf3-3ca9-48c3-b6c7-ba148aff51c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:41.797693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9e779bf3-3ca9-48c3-b6c7-ba148aff51c7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:41.798913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9e779bf3-3ca9-48c3-b6c7-ba148aff51c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:41.898170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 729/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:42.201830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:42.202109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:42.202400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:42.203731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 744/1473] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 17 19:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:42.208691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9b332-edfb-4139-924f-24df9cc4fef5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:42.209001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9b332-edfb-4139-924f-24df9cc4fef5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:42.209273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9b332-edfb-4139-924f-24df9cc4fef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:42.290417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4ba9b332-edfb-4139-924f-24df9cc4fef5 None None] MFA Rules not processed for user `9f03a7d5287c48e4821d308767f4c141`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:42.291644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4ba9b332-edfb-4139-924f-24df9cc4fef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:42.314425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:42.370654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 730/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 17 19:57:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:57:42.378342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:42.379900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:42.504432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa04bd-d023-4d95-a65d-4c7a62f6e6b7 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:42.582638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:42.583247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:42.583521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:42.583785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:42.585381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:42.642102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:42.700301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:42.702551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80488dcb-ec28-439c-a4a8-f549239ee30c service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:42.725066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 745/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:44.674383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:44.674917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:44.674917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:44.676305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 731/1476] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:44.682548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f83ab-3671-4876-9b44-e1a61e598a3d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:44.683250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f83ab-3671-4876-9b44-e1a61e598a3d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:44.683505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f83ab-3671-4876-9b44-e1a61e598a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:44.757040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-184f83ab-3671-4876-9b44-e1a61e598a3d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:44.758286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-184f83ab-3671-4876-9b44-e1a61e598a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:44.862420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 746/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:45.137662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:45.139116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:45.280421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898958-6289-4e1c-a129-1569d72e2176 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:45.332686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:45.379967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:45.380573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:45.380846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:45.381277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:45.381954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:45.383715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:57:45.383715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 19:57:45.383715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 19:57:45.384839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e29c5dd-1cf5-4637-a243-2ade337f88c0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:45.403853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 732/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 17 19:57:45 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 19:57:47.308837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:47.309687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:47.309687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:47.311723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 747/1479] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:47.320398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa727b-9c8b-4952-ac9d-05104d9a7ae9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:47.321201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa727b-9c8b-4952-ac9d-05104d9a7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:47.321379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa727b-9c8b-4952-ac9d-05104d9a7ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:47.394143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-90fa727b-9c8b-4952-ac9d-05104d9a7ae9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:47.395581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-90fa727b-9c8b-4952-ac9d-05104d9a7ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:47.513730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 733/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:47.810785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:47.812714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:47.823207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:47.968534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0070a30-8ef3-46c1-8dd8-fcb86534574d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:48.053002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:48.089623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:48.090161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:48.090431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:48.090691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:48.092291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:48.186795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:48.188419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b51c1e-a772-45bc-a9d8-f08cb5a01eec service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:48.206633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 748/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:50.314264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:50.314825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:50.314825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:50.316049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 734/1482] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:50.325051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc866-c7f5-4199-b5ec-790b907759b5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:50.325788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc866-c7f5-4199-b5ec-790b907759b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:50.326144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bc866-c7f5-4199-b5ec-790b907759b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:50.403510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e72bc866-c7f5-4199-b5ec-790b907759b5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:50.405252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e72bc866-c7f5-4199-b5ec-790b907759b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:50.479725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:50.533403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 749/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:50.824516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:50.827040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:50.856688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:50.944374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392a625-d4dd-42e4-b6dd-6e9dd81e6f3d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:51.019532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:51.020379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:51.020818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:51.021108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:51.023720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:51.104340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:51.145056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:51.146529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3569f-8f51-4404-b2ee-85d05a992110 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:51.165101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 735/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:53.156800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:53.157426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:53.157426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:53.158747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 750/1485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:53.165174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4cd-2cd3-41f9-b2d6-dbb7fda93fda None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:53.165978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4cd-2cd3-41f9-b2d6-dbb7fda93fda None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:53.166135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4cd-2cd3-41f9-b2d6-dbb7fda93fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:53.248943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1f57c4cd-2cd3-41f9-b2d6-dbb7fda93fda None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:53.250095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1f57c4cd-2cd3-41f9-b2d6-dbb7fda93fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:53.354229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 736/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:53.637379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:53.638965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:53.747531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bdb09-fe18-4c3e-95c2-a4852d20f340 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:53.828347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:53.830155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:53.830745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:53.831157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:53.831286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:53.833672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:53.958762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:53.960235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5276b2-6eeb-4275-94d7-d3c47e2ba564 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:53.980175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 751/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:55.999890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:56.000766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:56.000766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:56.002958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 737/1488] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:56.009153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d673ec-da4d-4f94-8c1b-7e03cce88b5f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:56.009629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d673ec-da4d-4f94-8c1b-7e03cce88b5f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:56.009738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d673ec-da4d-4f94-8c1b-7e03cce88b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:56.082990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a6d673ec-da4d-4f94-8c1b-7e03cce88b5f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:56.084369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a6d673ec-da4d-4f94-8c1b-7e03cce88b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:56.137688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:56.188542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 752/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:57:56.484640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:56.486223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:56.530492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:56.606178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b343ed-bec3-43eb-92d5-9ff814de705e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:56.687150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:56.687768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:56.688088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:56.688355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:56.690013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:56.773001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:56.806280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:56.808376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3faa15-0151-4cca-9c5d-33b174d83def service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:56.837402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 738/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:57:58.983051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:58.983814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:58.983814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:58.985001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 753/1491] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 17 19:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:57:58.991346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030d4c-8b3e-4dd8-9c26-91521b1ad786 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:58.991831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030d4c-8b3e-4dd8-9c26-91521b1ad786 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:58.991949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030d4c-8b3e-4dd8-9c26-91521b1ad786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:59.067276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4d030d4c-8b3e-4dd8-9c26-91521b1ad786 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:57:59.068503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d030d4c-8b3e-4dd8-9c26-91521b1ad786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:59.170314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 739/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 17 19:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 19:57:59.479144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:57:59.480764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:59.588682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4855b0-3726-4e03-ac69-65a339135d6c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:57:59.673296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:57:59.673947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:57:59.674226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:57:59.674491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:57:59.676117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:57:59.686111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:57:59.792212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:57:59.793590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715cecf-4eba-4e9e-a7fc-e628fe7cf3ba service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:57:59.813682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 754/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:58:09.168406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:58:09.810982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=50 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:17.526516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:17.527160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:17.527160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:17.606637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:59:17.607844 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:17.708342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 740/1494] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:59:17.716835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:17.717374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:17.717374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:17.797019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:59:17.798186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:17.902586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 755/1495] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 19:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:59:17.907592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:17.908821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:17.921327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa92f0-2ed6-4151-ac47-a55ed6cd1179 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:17.987129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:18.026719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.027339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.027613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.027871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.028521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 19:59:18.028818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.029902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.029902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 19:59:18.029902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 19:59:18.030744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:18.042729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 741/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 19:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 19:59:18.051837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:18.061472 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fc54e-3184-40ad-8176-82487a4d42d5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:18.130260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:18.166925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.167528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.167796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.168056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.168673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.170548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.170548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 19:59:18.170548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 19:59:18.172094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:18.235365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 756/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 19:59:18 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:59:18.239855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:18.241405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:18.276256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341b1ad-409b-4d9e-b158-446fefc59dbe admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:18.374547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.375377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.375719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.376077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.377073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.378597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.378597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 19:59:18.378597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 19:59:18.380560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:18.429209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 742/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 19:59:18 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 19:59:18.434791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:18.436345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:18.501485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0b4e4-6cc6-4403-a9f3-1086471e3399 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:18.593201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:18.605944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.606559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.606830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.607096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.607881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.608920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.608920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 19:59:18.608920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 19:59:18.609847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:18.624842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 757/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 19:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:59:18.631037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:18.633167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:18.650488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada9e2-d59a-4c6a-b5ca-26dbcc3a6713 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:18.677167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:18.760477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.761101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.761378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.761690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.762328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.763345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.763345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 19:59:18.763345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 19:59:18.764229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeeebf9-5847-47e0-b496-6a5dccbfe9db admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:18.777525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 743/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 19:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 19:59:18.782030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:18.783583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:18.818903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ab80a-8d6c-43dc-ad85-4490cd927b7c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:18.922401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:18.922970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:18.923237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:18.923501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] PATH_INFO: `/v3/projects/02f968439edd4b5b83cc80abd3207727/users/ff7e776d33554d72b8ac1df85ddb3566/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:18.947672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f968439edd4b5b83cc80abd3207727, user_id=ff7e776d33554d72b8ac1df85ddb3566, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:18.949216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:18.949216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 19:59:18.949216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 19:59:18.952285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efac85-bb3d-48a0-8775-183561367105 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:19.110082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 758/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 19:59:18 2022] PUT /identity/v3/projects/02f968439edd4b5b83cc80abd3207727/users/ff7e776d33554d72b8ac1df85ddb3566/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:59:19.122656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf2a5-9b51-4c76-aa07-0d84afaf5d4a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:19.122945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf2a5-9b51-4c76-aa07-0d84afaf5d4a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:19.123214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cf2a5-9b51-4c76-aa07-0d84afaf5d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:19.181012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-eb4cf2a5-9b51-4c76-aa07-0d84afaf5d4a None None] MFA Rules not processed for user `ff7e776d33554d72b8ac1df85ddb3566`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:59:19.182177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-eb4cf2a5-9b51-4c76-aa07-0d84afaf5d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:19.264200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 744/1502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 19:59:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:59:19.278363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:19.278639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:19.278905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:19.280231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 759/1503] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 17 19:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:59:19.285389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ee3d-b625-423d-b2bd-87918bc3be87 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:19.285714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ee3d-b625-423d-b2bd-87918bc3be87 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:19.285983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946ee3d-b625-423d-b2bd-87918bc3be87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:19.319671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:19.366328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e946ee3d-b625-423d-b2bd-87918bc3be87 None None] MFA Rules not processed for user `c97be7f3685b44cfa26aeae0f5b5d931`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:59:19.367609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e946ee3d-b625-423d-b2bd-87918bc3be87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:19.462122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 745/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 17 19:59:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:59:19.470551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:19.473023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:19.534922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:19.593015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df359-f0b7-4bdd-8af4-6d90c6967fba None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:19.672414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:19.673019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:19.673291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:19.673587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:19.675192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:19.780141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:19.781476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:19.785063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:19.801351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 760/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:59:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:59:19.849059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405121a-cd2d-4f9c-accf-6c357c6db68f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:19.849478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405121a-cd2d-4f9c-accf-6c357c6db68f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:19.849668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405121a-cd2d-4f9c-accf-6c357c6db68f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:19.850946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 746/1506] 10.222.0.18 () {52 vars in 830 bytes} [Wed Aug 17 19:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 19:59:19.854685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:19.855958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:19.964643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9e2cd-a96f-4455-97dc-7d0b3b066c6e service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:20.071182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:20.071722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:20.072000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:20.072258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] PATH_INFO: `/v3/users/ff7e776d33554d72b8ac1df85ddb3566` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:20.085213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff7e776d33554d72b8ac1df85ddb3566)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:20.086359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:20.086359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 19:59:20.086359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 19:59:20.087557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:20.099949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:20.169814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff7e776d33554d72b8ac1df85ddb3566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:20.185521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff7e776d33554d72b8ac1df85ddb3566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:20.208668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff7e776d33554d72b8ac1df85ddb3566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:20.219794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff7e776d33554d72b8ac1df85ddb3566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:20.220388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff7e776d33554d72b8ac1df85ddb3566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:20.226981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 761/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 19:59:19 2022] DELETE /identity/v3/users/ff7e776d33554d72b8ac1df85ddb3566 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 19:59:20.231637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:20.231990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:20.232326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:20.288565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 19:59:20.289817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:20.369390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:20.382794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 747/1508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 19:59:20.392671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:20.393959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:20.501169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae84521-f248-465e-89f4-01e4757e5d13 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:20.557860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:20.578973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:20.579612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:20.579883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:20.580145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:20.582006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:20.694103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:20.695669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32e1fe-15c8-487a-a395-4d3e6e76b0c6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:20.714576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 762/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 19:59:21.145117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 19:59:21.146420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 19:59:21.158190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab60b3-fbff-4041-9442-0cd3ff924ac2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 19:59:21.254241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 19:59:21.254766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 19:59:21.255038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 19:59:21.255301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] PATH_INFO: `/v3/projects/02f968439edd4b5b83cc80abd3207727` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 19:59:21.265294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f968439edd4b5b83cc80abd3207727)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 19:59:21.266348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 19:59:21.266348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 19:59:21.266348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 19:59:21.267540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 19:59:21.331478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f968439edd4b5b83cc80abd3207727', 'request_id': 'req-8211b407-a141-42ae-a235-73c8fa2e3839'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:21.337602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f968439edd4b5b83cc80abd3207727', 'request_id': 'req-8211b407-a141-42ae-a235-73c8fa2e3839'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:21.348668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f968439edd4b5b83cc80abd3207727', 'request_id': 'req-8211b407-a141-42ae-a235-73c8fa2e3839'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:21.378935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] The token cache is being invalidate because project 02f968439edd4b5b83cc80abd3207727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 19:59:21.379214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:21.379633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8211b407-a141-42ae-a235-73c8fa2e3839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 19:59:21.381023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 748/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 19:59:21 2022] DELETE /identity/v3/projects/02f968439edd4b5b83cc80abd3207727 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 19:59:30.711851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=62 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 19:59:31.376945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:00:48.979775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:48.980427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:48.980427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:48.982779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 20:00:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:00:48.990822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c8433-6468-4905-9c08-3a1f7b3c720a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:48.991243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c8433-6468-4905-9c08-3a1f7b3c720a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:48.991353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c8433-6468-4905-9c08-3a1f7b3c720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:49.077813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-de3c8433-6468-4905-9c08-3a1f7b3c720a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:00:49.079092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-de3c8433-6468-4905-9c08-3a1f7b3c720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:00:49.201398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:00:49.352810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:00:49.356187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:00:49.495153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab47ac0-b16a-4a14-91b3-46971d785fa1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:00:49.501416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:00:49.502047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:49.502321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:49.502585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:49.511228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:00:49.512819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fcd07a-1dbd-4b03-bebb-584fbb1645bb service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:00:49.522373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 764/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:00:50.351163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4687b-084c-4dee-a502-6814bdd32874 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:50.352022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4687b-084c-4dee-a502-6814bdd32874 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:50.352022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4687b-084c-4dee-a502-6814bdd32874 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:50.354114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 20:00:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:00:50.360163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b148d7-37ad-4c69-acee-7bd5de0136c5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:50.360898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b148d7-37ad-4c69-acee-7bd5de0136c5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:50.361315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b148d7-37ad-4c69-acee-7bd5de0136c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:50.387504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20b148d7-37ad-4c69-acee-7bd5de0136c5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:00:50.388711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20b148d7-37ad-4c69-acee-7bd5de0136c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:00:50.397899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:00:50.551313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4687b-084c-4dee-a502-6814bdd32874 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:00:50.567918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4687b-084c-4dee-a502-6814bdd32874 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:00:50.574125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:00:50.574729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:50.575007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:50.575273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:50.584390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:00:50.586020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7e1cf-eebb-4cd6-b91d-ef3ed6b6c68c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:00:50.596043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 751/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:00:50.973146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46ce9-9145-4320-a76b-d7e8f7f22fae None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:50.973383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46ce9-9145-4320-a76b-d7e8f7f22fae None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:50.973643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd46ce9-9145-4320-a76b-d7e8f7f22fae None None] PATH_INFO: `/v3` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:50.974944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 766/1517] 10.222.0.46 () {54 vars in 925 bytes} [Wed Aug 17 20:00:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:00:50.982472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107dae-5114-4203-bec9-16d1bd56a008 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:00:50.982759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107dae-5114-4203-bec9-16d1bd56a008 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:00:50.983028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5107dae-5114-4203-bec9-16d1bd56a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:00:51.045114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d5107dae-5114-4203-bec9-16d1bd56a008 None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:00:51.046729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d5107dae-5114-4203-bec9-16d1bd56a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:00:51.064234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:00:51.131877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 752/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Aug 17 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:00.277486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45f22-8afe-4f8b-8590-d1fdb8975087 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:00.278413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45f22-8afe-4f8b-8590-d1fdb8975087 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:00.278413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb45f22-8afe-4f8b-8590-d1fdb8975087 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:00.280380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 767/1519] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 17 20:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:00.288350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b81e0-45ad-4b5d-880e-62cfa8e19a0b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:00.288811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b81e0-45ad-4b5d-880e-62cfa8e19a0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:00.288919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b81e0-45ad-4b5d-880e-62cfa8e19a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:00.310040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0e0b81e0-45ad-4b5d-880e-62cfa8e19a0b None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:00.311223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e0b81e0-45ad-4b5d-880e-62cfa8e19a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:00.320461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 753/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 17 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:01:00.328064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45f22-8afe-4f8b-8590-d1fdb8975087 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:00.342877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb45f22-8afe-4f8b-8590-d1fdb8975087 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:00.348399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:00.348981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:00.349252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:00.349510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:00.358383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:00.359948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259025-3d4b-4dce-b503-9911ecc0e19f service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:00.370009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 768/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:01:01.839290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4065152-ddfa-442f-9dc6-efc31129bcb5 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:01.840250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4065152-ddfa-442f-9dc6-efc31129bcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:01.840250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4065152-ddfa-442f-9dc6-efc31129bcb5 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:01.842349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 20:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 17 20:01:01.847409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f86b6-7212-4fbc-867c-0535ad90ca80 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:01.847857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f86b6-7212-4fbc-867c-0535ad90ca80 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:01.847970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f86b6-7212-4fbc-867c-0535ad90ca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:01.869223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c77f86b6-7212-4fbc-867c-0535ad90ca80 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:01.870442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c77f86b6-7212-4fbc-867c-0535ad90ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:01.878834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:02.034220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4065152-ddfa-442f-9dc6-efc31129bcb5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:02.055387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4065152-ddfa-442f-9dc6-efc31129bcb5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:02.061719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:02.062321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:02.062593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:02.062863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:02.071024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:02.072634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9058897-d341-4838-b3a0-230b76fc0d67 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:02.081914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 755/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:01:02.898349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761cba0-c0c4-4248-bb29-82836686b8ef None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:02.899249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761cba0-c0c4-4248-bb29-82836686b8ef None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:02.899249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761cba0-c0c4-4248-bb29-82836686b8ef None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:02.901379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 17 20:01:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:02.906528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b7122-c416-439f-83e3-7ae48116a984 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:02.906967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b7122-c416-439f-83e3-7ae48116a984 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:02.907103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b7122-c416-439f-83e3-7ae48116a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:02.928728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-816b7122-c416-439f-83e3-7ae48116a984 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:02.929953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-816b7122-c416-439f-83e3-7ae48116a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:02.942159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 17 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:03.096945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761cba0-c0c4-4248-bb29-82836686b8ef None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:03.116324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761cba0-c0c4-4248-bb29-82836686b8ef None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:03.123699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:03.124290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:03.124556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:03.124819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:03.133345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:03.135069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9995a6d-9654-4f9f-aaac-cb206ca42f35 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:03.145560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 771/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:01:07.002893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:07.003494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:07.003494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:07.005573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 20:01:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:07.012674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fded7e4-6b27-40aa-99bb-428c7306f50d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:07.013118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fded7e4-6b27-40aa-99bb-428c7306f50d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:07.013224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fded7e4-6b27-40aa-99bb-428c7306f50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:07.035309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6fded7e4-6b27-40aa-99bb-428c7306f50d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:07.036511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6fded7e4-6b27-40aa-99bb-428c7306f50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:07.046149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:07.343103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:07.345213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:07.426311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e691c-ee2c-4a64-8554-a3520ea2a1e0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:07.431156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:07.431769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:07.432042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:07.432283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:07.442050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:07.443485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3319845-2112-4919-8a88-5cc2bb91ba74 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:07.452835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 758/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:01:09.301152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80463-ce1f-4bbb-b6ac-ae0f291bda8c None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:09.302152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80463-ce1f-4bbb-b6ac-ae0f291bda8c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:09.302152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80463-ce1f-4bbb-b6ac-ae0f291bda8c None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:09.304188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 20:01:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:09.312255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b2da-a200-4e70-8f94-9866965016d6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:09.312718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b2da-a200-4e70-8f94-9866965016d6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:09.312814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b2da-a200-4e70-8f94-9866965016d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:09.334130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4695b2da-a200-4e70-8f94-9866965016d6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:09.335329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4695b2da-a200-4e70-8f94-9866965016d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:09.343909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 20:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:01:09.628152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80463-ce1f-4bbb-b6ac-ae0f291bda8c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:09.641110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80463-ce1f-4bbb-b6ac-ae0f291bda8c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:09.644873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:09.645478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:09.645795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:09.646061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:09.656085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:09.657575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3d0a9-5b75-4c04-a3f7-2698e08cc254 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:09.667243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 774/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:01:13.137512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:13.138144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:13.138144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:13.139604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 20:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:13.147907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cfd6b-9d5a-45eb-b042-d8df892df2a6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:13.148737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cfd6b-9d5a-45eb-b042-d8df892df2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:13.148737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cfd6b-9d5a-45eb-b042-d8df892df2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:13.158207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:01:13.172844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-478cfd6b-9d5a-45eb-b042-d8df892df2a6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:13.174121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-478cfd6b-9d5a-45eb-b042-d8df892df2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:13.186834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 20:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:13.738780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:13.740734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:13.814835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:01:13.835104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedba585-0786-4697-8c6f-8f1661058069 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:13.838899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:13.839513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:13.839789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:13.840063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:13.849634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:13.851116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88cda4-d377-4f62-b20c-83efdd5479af service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:13.860745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 761/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:01:15.788294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:15.789196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:15.789196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:15.791332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 17 20:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:01:15.800938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d65e3-97f0-43c5-b5ae-371cb3a9915d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:15.801644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d65e3-97f0-43c5-b5ae-371cb3a9915d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:15.801798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d65e3-97f0-43c5-b5ae-371cb3a9915d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:15.827518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a69d65e3-97f0-43c5-b5ae-371cb3a9915d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:01:15.828721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a69d65e3-97f0-43c5-b5ae-371cb3a9915d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:15.839206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 17 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:01:16.385504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:01:16.387623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:01:16.402657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b786bbd-0a51-43ee-9700-5179810f8b0d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:01:16.407207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:01:16.408175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:01:16.408453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:01:16.408721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:01:16.420464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:01:16.422220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27137566-8ac2-492a-8719-88135b356107 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:01:16.432850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 777/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:01:25.822656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=11 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:01:26.428100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:41.524311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.524958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.524958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.554448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.555724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.569833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 763/1540] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:41.580634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ee60-b427-4a60-833f-251a86858072 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.581721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ee60-b427-4a60-833f-251a86858072 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.582196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6ee60-b427-4a60-833f-251a86858072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.613243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20a6ee60-b427-4a60-833f-251a86858072 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.614863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20a6ee60-b427-4a60-833f-251a86858072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.623780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 778/1541] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:41.629035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:41.643268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c12551-f8e7-4081-b802-67aaf65a3dce None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:41.653050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:41.654203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.654655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.655096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.655931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:41.656912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:41.658985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:41.672211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3242ce-83cf-4d7b-a34b-c3365f8a1a99 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.672828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3242ce-83cf-4d7b-a34b-c3365f8a1a99 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.673282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3242ce-83cf-4d7b-a34b-c3365f8a1a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.685397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 764/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:41.707394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4c3242ce-83cf-4d7b-a34b-c3365f8a1a99 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.709444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4c3242ce-83cf-4d7b-a34b-c3365f8a1a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.712448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:41.724756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99143006-d471-46b9-a9e4-c1ad96872afa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:41.729560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 779/1543] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:41.735535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1561f-814b-4a4d-92b3-fe5adfd12186 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.735833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1561f-814b-4a4d-92b3-fe5adfd12186 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.736074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1561f-814b-4a4d-92b3-fe5adfd12186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.737549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:41.738442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.739010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.739490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.740302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:41.741504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:41.741504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 20:02:41.741504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 20:02:41.743264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518d126-6632-4c2c-b262-ebea224d1bc5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:41.774254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7bd1561f-814b-4a4d-92b3-fe5adfd12186 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.775531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7bd1561f-814b-4a4d-92b3-fe5adfd12186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.795770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 780/1544] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:02:41.797637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.799114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.799114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.818806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 765/1545] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:02:41.820121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.820411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.820686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.840952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.842876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.857557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.859011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.934265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 781/1546] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:41.935873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:41.946765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b44-119a-4336-adb5-f288f49be583 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:41.951644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:41.952244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.952518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.952843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.953529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:41.953853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:41.954142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 766/1547] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:41.954943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:41.954943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 20:02:41.954943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 20:02:41.955183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:41.955892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fbac34-d922-44d8-93fe-9faa5b4bea4e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:41.964441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b92fa-7ce2-466e-ac4c-90973f8f86ed None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:41.968248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 782/1548] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:41.969245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:41.969645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563fd90-acb0-42a1-941f-31521d37db3e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.969885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.970005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563fd90-acb0-42a1-941f-31521d37db3e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.970168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.970275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563fd90-acb0-42a1-941f-31521d37db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.970432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:41.971050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:41.972962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec29980-acb0-49db-ad8d-a2586d1a0c56 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:41.990689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5563fd90-acb0-42a1-941f-31521d37db3e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:41.992234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5563fd90-acb0-42a1-941f-31521d37db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:41.999372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 767/1549] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:41.999372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8528-0333-44a2-ac67-cedbef417357 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:41.999372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8528-0333-44a2-ac67-cedbef417357 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:41.999372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8528-0333-44a2-ac67-cedbef417357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.004298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 783/1550] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.005524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36ba8-1c20-4ee3-85bb-066a7ad6dd43 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.005880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36ba8-1c20-4ee3-85bb-066a7ad6dd43 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.006102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d36ba8-1c20-4ee3-85bb-066a7ad6dd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.018757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a31b8528-0333-44a2-ac67-cedbef417357 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.020030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a31b8528-0333-44a2-ac67-cedbef417357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.026615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e8d36ba8-1c20-4ee3-85bb-066a7ad6dd43 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.027731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e8d36ba8-1c20-4ee3-85bb-066a7ad6dd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.028156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 768/1551] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.032405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d4-3441-461c-a871-6494cd00e192 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.032405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d4-3441-461c-a871-6494cd00e192 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.032405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d4-3441-461c-a871-6494cd00e192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.042058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 784/1552] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.043311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.043652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.043975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.055380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b5b546d4-3441-461c-a871-6494cd00e192 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.056580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b5b546d4-3441-461c-a871-6494cd00e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.063868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.064621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 769/1553] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.065041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.065761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.066039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.066306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.075539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 785/1554] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.076511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.085591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.085982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34899b-373f-4eb3-a685-29081aa46e51 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.086793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.090809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.091453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.091729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.091996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.092625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.094556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c59378d-cbe4-4b75-8c43-407587e9c415 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.094808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 770/1555] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.095630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.105102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ec85-131c-4638-863c-207346d2c3a8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.109838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.110424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.110699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.110970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.111605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.113208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b96a95-654c-46ad-8d9a-2b0d593e3d96 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.124011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 771/1556] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:42.128064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 786/1557] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.128227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.128227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.128227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.128530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd316-3f7e-4910-91f3-b264842f84f8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.128530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd316-3f7e-4910-91f3-b264842f84f8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.128530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd316-3f7e-4910-91f3-b264842f84f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.146326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ab7dd316-3f7e-4910-91f3-b264842f84f8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.147531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ab7dd316-3f7e-4910-91f3-b264842f84f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.148482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.149702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.207388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 787/1558] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.208652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e3fc-a983-40e0-a08a-c29a438f3ea3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.208934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e3fc-a983-40e0-a08a-c29a438f3ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.209132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 772/1559] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.209487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9e3fc-a983-40e0-a08a-c29a438f3ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.213633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.218232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.224568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e0c36-0a02-4ae2-ad9a-ba994df29d82 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.230008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.230150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5cc9e3fc-a983-40e0-a08a-c29a438f3ea3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.230620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.230896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.231167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.231318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5cc9e3fc-a983-40e0-a08a-c29a438f3ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.231816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.232113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.233750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.240939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 788/1560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.242159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.242445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.242709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.246111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 773/1561] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.247240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.260963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81bf89-2049-4ada-9cf7-8819fb370d26 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.266298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.266897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.267172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.267451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.268112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.268265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.268409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.269445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.270031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.279557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 789/1562] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.280511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.288062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 774/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.288062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.293215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ecbb7-4987-4ac9-b669-da177dd6fd6b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.294276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d6547-25e7-485c-bba8-b4d81082a352 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.298413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.299015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.299221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.299339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.299542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.299797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.300121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.300227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.300390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.301003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.301861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.302904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.313268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 790/1564] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:42.313577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.319422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.325817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccfadd-10c3-479c-bd06-289e89778070 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.330650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 775/1565] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.330650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.331201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.332193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.332640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.333154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.333905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.334213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.335987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.338106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907131af-cf3d-467e-b70c-f69fbc9764c8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.343170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.343853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.344127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.344390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.345026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.345321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.346911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.349147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 791/1566] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.350041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.361272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253df3-af11-4c5a-a582-c8d47e58383f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.365156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 776/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.366112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.366261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.366813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.367090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.367357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.367999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.368303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.369948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.377748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc393-36e9-4c1b-bd1b-2fa0c632c0e2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.382129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 792/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.382941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.385878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.386563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.386907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.387240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.388127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.391038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.392614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a403fe4-2a02-4da2-b91b-a3eef4d51a36 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.397354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.398016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.398294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.398564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.399216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:42.399516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.401129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.414488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 793/1569] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:42.415626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.427278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e161acc-687b-4003-bb06-532f1d7f4f62 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.428956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 777/1570] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.429891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.487072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.487680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.487950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.488259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.488878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.490846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.497061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409fdd4a-9a82-4392-aeeb-20a2299b9734 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.501858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.502725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.502906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.503175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] PATH_INFO: `/v3/projects/56d236fd14cd49168a3842369badc524/users/454c2bf24ed94f82b453df36e6e8eac1/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.519075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56d236fd14cd49168a3842369badc524, user_id=454c2bf24ed94f82b453df36e6e8eac1, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.519855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 794/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.520172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:42.520172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 20:02:42.520172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 20:02:42.520674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.522355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.536479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 778/1572] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:42 2022] PUT /identity/v3/projects/56d236fd14cd49168a3842369badc524/users/454c2bf24ed94f82b453df36e6e8eac1/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:42.537207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.586171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cac597-1055-4a24-901c-5197d18c5031 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.601987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de513f-322a-4ee4-9be6-752b3987b510 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.606980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.607595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.607939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.608219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.608885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.610845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.622283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.638649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.643394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 779/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.644371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.647345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.648139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.648448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.648977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.649741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.651773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.662168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 795/1574] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:42.666047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.677188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f40c-57e0-467f-97e2-e69c3b6bec44 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.708088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcc48c-184b-4c4c-baac-9b8736a30ca6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.714694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.715306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.715604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.715873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.716538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.718805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.736909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.737644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.737902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.738161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.738807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.740761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.748083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 780/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.749026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.773847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 796/1576] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.773847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.818067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9a25e-d7bf-4584-bb13-518ee57573e1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.832478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aeda4e-c790-475d-b928-aa0f4b84e17e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.837148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.837788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.838060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.838327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.838952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.840072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:42.840072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 20:02:42.840072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 20:02:42.841488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.865343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 797/1577] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.866191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.876064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6dea-d0cc-4914-bf59-a02ff7661bf3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.876330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.876857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.877138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.877410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.878076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.879995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.881192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.881812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.882086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.882349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.882966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.884891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09a032-a9dd-48f0-8a74-0b86cc25be1b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.908627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 798/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.909779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 781/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:42.910104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.910428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.910702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.913497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:42.924022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.929787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:42.931144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:42.975925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:42.979212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3691c9-37e5-4aa1-8066-c37dbdf42297 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:42.984910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:42.985559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:42.985957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:42.986270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:42.986891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:42.988927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:42.993202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 799/1580] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:42.994740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.006714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c33ca3-267d-4240-b4f7-f2db52c4da91 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.012035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.012709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.013052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.013379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.014120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.014816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 782/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:42 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:43.015668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.015909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.026456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd4a31-efa7-4793-b7f1-9f1dda7285cb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.029698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 800/1582] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.030861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.031174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.031783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.032059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.032329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.032956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.034937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.040453 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ec2c7-fe87-4564-8692-cc61c96a45c9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.045205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.045931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.046202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.046796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.047411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.049346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.058712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 783/1583] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:43.062922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.073508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 801/1584] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/users => generated 286 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:43.073508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.073912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f182189-3d4c-414d-8b06-565c1101a47d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.078073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.078654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.078941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.079207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.079829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.081417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.083294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f601645-06dc-407d-87fd-1437d3a5b6fc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.087863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.088447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.088717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.088967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.089676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.091309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.094108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 784/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.094880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.102466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 802/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.103258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.104166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eba6a4-f709-4ceb-9148-68c14b445666 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.109653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.110263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.110540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.110808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.111434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.113369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.136538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 785/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:43.137358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.147000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ef99-4130-4c8b-8dbe-82bf57b9b684 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.152722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.153305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.153580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.153857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.154483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.156065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.160791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623d602-ede9-4394-b105-42f5d79b3293 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.164354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.164955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.165226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.165489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.166335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 786/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.167112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.178149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.178754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f248bce-a594-4bca-aa39-4a2c728da2c5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.179582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.183473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.184106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.184384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.184653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] PATH_INFO: `/v3/projects/83184ecf125d4a99894aad39a9046b12/users/20968c5893fe42e79547c978d6873845/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.188980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 803/1589] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:43.190543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.199699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=83184ecf125d4a99894aad39a9046b12, user_id=20968c5893fe42e79547c978d6873845, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.200179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a0c9d-46bc-461c-8c37-e811127f2f72 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.202636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.205467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.206110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.206388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.206658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.207348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.208968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.219064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 787/1590] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/83184ecf125d4a99894aad39a9046b12/users/20968c5893fe42e79547c978d6873845/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.219652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.223031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 804/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.224432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.239055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4d4a-39d1-4dde-b8fe-2ffc22735c6f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.289319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e2b42-d75e-4d7d-abe8-e2549018eaec admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.294574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.295196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.295473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.295743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.296786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.298430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.304475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.305092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.305363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.305655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.306301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.307893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.312541 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 788/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.313508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.319483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 805/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.320307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.326672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbf9bb-8c2b-4d3c-9a8c-01547dcb247e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.332691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.332883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.333229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.333459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2a21-0512-4337-bf88-fd9c27c0a355 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.334041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.334439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.336230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.339363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.340133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.340551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.340984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.341929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.343794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.347947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 789/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.348791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.354107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:43.359276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 806/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.359276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.362536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f5ad9-6052-4079-b035-6168fec9b4b5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.369112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f196d-cf2e-4f6d-a0c6-4c2584612323 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.369268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.369970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.370584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.370938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.371653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.373728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.374221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.375033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.375304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.375571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.376270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.378100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a2cf6-d339-4f85-bad5-bbb84f37ba43 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.394176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 807/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.394176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829465-0925-4497-a77e-e2661a093522 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.394176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829465-0925-4497-a77e-e2661a093522 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.394176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829465-0925-4497-a77e-e2661a093522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.402366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 790/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.402366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.413264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882aa0-453a-4641-83b0-f0616597a0e6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.418756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.419556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.419881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.420232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.421033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.423325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.437638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 791/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.437812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ab829465-0925-4497-a77e-e2661a093522 None None] MFA Rules not processed for user `20968c5893fe42e79547c978d6873845`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:43.437914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.438045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ab829465-0925-4497-a77e-e2661a093522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:43.441409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:43.451865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8622a-c5d0-461e-bb45-0aa952ce053f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.457346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.458084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.458455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.458728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] PATH_INFO: `/v3/projects/10570b24f117432d9b887de535ea8764/users/15a2ed7e5b124aa5ba934b91e1a66afd/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.477841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10570b24f117432d9b887de535ea8764, user_id=15a2ed7e5b124aa5ba934b91e1a66afd, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.481731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.498818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 792/1599] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/10570b24f117432d9b887de535ea8764/users/15a2ed7e5b124aa5ba934b91e1a66afd/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.499629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.519904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 808/1600] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:43.520987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829465-0925-4497-a77e-e2661a093522 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.532724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829465-0925-4497-a77e-e2661a093522 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.593749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590a902-778e-4494-860e-ea76ed11f108 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.599813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.600500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.600782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.601058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] PATH_INFO: `/v3/projects/db932807b4634440ba61a82446e82874/users/a5aece8a7f2342cfabd9a3f8707a4470/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.615115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.615825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.616574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.616574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.617361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.617851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db932807b4634440ba61a82446e82874, user_id=a5aece8a7f2342cfabd9a3f8707a4470, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.620234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.620596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.631847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 809/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:43.632783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.639319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 793/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/db932807b4634440ba61a82446e82874/users/a5aece8a7f2342cfabd9a3f8707a4470/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.643893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.644010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc250e7-bdd3-42e3-abbd-8cc74a77c57b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.714205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.715198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.715476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.715747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] PATH_INFO: `/v3/projects/c0c02701e05249459322208ed51226f4/users/115ba18c2f034f9ea698e5a426846175/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.720194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e408806-ee38-43a8-a7a3-1c7f6674d260 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.727239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.728834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.729321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.729833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] PATH_INFO: `/v3/projects/5433b5d7443545249b9329f1ce9e3e02/users/b095ab3c72b14c67adf61ab4189e4892/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.731459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c02701e05249459322208ed51226f4, user_id=115ba18c2f034f9ea698e5a426846175, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.734236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.750317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 810/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/c0c02701e05249459322208ed51226f4/users/115ba18c2f034f9ea698e5a426846175/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.751393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.751516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5433b5d7443545249b9329f1ce9e3e02, user_id=b095ab3c72b14c67adf61ab4189e4892, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.754540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a27f2-d65a-4db3-8e79-78d0f40c1d04 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.772435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 794/1604] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/5433b5d7443545249b9329f1ce9e3e02/users/b095ab3c72b14c67adf61ab4189e4892/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.776325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c59ee6-b2b8-4fed-bde1-a5bf2993801c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.776325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c59ee6-b2b8-4fed-bde1-a5bf2993801c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.777056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c59ee6-b2b8-4fed-bde1-a5bf2993801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.783249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:43.813188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-16c59ee6-b2b8-4fed-bde1-a5bf2993801c None None] MFA Rules not processed for user `15a2ed7e5b124aa5ba934b91e1a66afd`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:43.814698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-16c59ee6-b2b8-4fed-bde1-a5bf2993801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:43.824092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadfb0-399f-4fdf-94a3-fd203f93292f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.825021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:43.894819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 795/1605] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:43.896208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513be198-dace-47c9-8829-17f3589e74fd None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.896507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513be198-dace-47c9-8829-17f3589e74fd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.896778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513be198-dace-47c9-8829-17f3589e74fd None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.899246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 796/1606] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 17 20:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:02:43.899443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513be198-dace-47c9-8829-17f3589e74fd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:43.908263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.909034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.909316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.909622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] PATH_INFO: `/v3/projects/897802aa26b441a896c49a5d1760f05f/users/74ef516c242341cbb287dacc83f1b78c/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.913232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513be198-dace-47c9-8829-17f3589e74fd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:43.920878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:43.924832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.925128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.925723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] PATH_INFO: `/v3/projects/da5173bee1cf4ec09240370892c5b9e3/users/3a40f55377b64f6f93784eefaf28152e/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.930476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897802aa26b441a896c49a5d1760f05f, user_id=74ef516c242341cbb287dacc83f1b78c, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.933598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8386fd1-ad3f-47db-9c10-ba5dc2a51348 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.952489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5173bee1cf4ec09240370892c5b9e3, user_id=3a40f55377b64f6f93784eefaf28152e, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:43.955441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa184414-4015-4710-bdad-c046f7f94027 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:43.956406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 811/1607] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/897802aa26b441a896c49a5d1760f05f/users/74ef516c242341cbb287dacc83f1b78c/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.958255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52195af-059a-462f-8225-b35d8f6170ea None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.958901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52195af-059a-462f-8225-b35d8f6170ea None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.959394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52195af-059a-462f-8225-b35d8f6170ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.980120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 797/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:43 2022] PUT /identity/v3/projects/da5173bee1cf4ec09240370892c5b9e3/users/3a40f55377b64f6f93784eefaf28152e/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:43.983891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d789-4390-48b9-89a7-11128030a406 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:43.983891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d789-4390-48b9-89a7-11128030a406 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:43.983891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052d789-4390-48b9-89a7-11128030a406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:43.987336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f52195af-059a-462f-8225-b35d8f6170ea None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:43.988853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f52195af-059a-462f-8225-b35d8f6170ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.020742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8052d789-4390-48b9-89a7-11128030a406 None None] MFA Rules not processed for user `115ba18c2f034f9ea698e5a426846175`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.022266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8052d789-4390-48b9-89a7-11128030a406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.056904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 812/1609] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.058704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.059238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.059732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.081384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.082645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.090959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 813/1610] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.095130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.105574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443727b-48b5-41b0-9c67-1396ae938e52 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.111401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.112094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.112430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.112766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.113507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.115842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b697d5-f65d-4a82-8bc8-14748147c4cd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.121896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 798/1611] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.123338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.123629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.123911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.160720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 814/1612] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/projects => generated 405 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:44.160720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca32e7-7f09-42c7-9b41-730043de7660 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.160720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca32e7-7f09-42c7-9b41-730043de7660 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.160720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca32e7-7f09-42c7-9b41-730043de7660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.202902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-20ca32e7-7f09-42c7-9b41-730043de7660 None None] MFA Rules not processed for user `74ef516c242341cbb287dacc83f1b78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.204804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-20ca32e7-7f09-42c7-9b41-730043de7660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.214273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.216177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.221723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:44.288217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:44.311172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 815/1613] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.312939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.313463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.313944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.319296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 799/1614] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.320646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.322330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.345312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.347295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.395938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e5e17-ba97-4846-ba86-7c35c3e7cef3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.400713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.401521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.402096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.402517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.455697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 816/1615] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.456517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.467804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b118cd-eedf-4d33-a833-f3c07767f119 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.471693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.472312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.472588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.472851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.487714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.489207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.499947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 817/1616] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:44.500927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.511573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01149f0-2d7e-4c52-976b-ecae23c0cdbc service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.515151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.516348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.516962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.517233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.517497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.518140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.520087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.520965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c3b0e-e9e9-4596-b4e8-01898bb9fca8 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.537386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 800/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:44.539112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd2b3-2130-4e47-9b02-a186f4c39917 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.539522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd2b3-2130-4e47-9b02-a186f4c39917 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.539896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd2b3-2130-4e47-9b02-a186f4c39917 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.541499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 801/1618] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 17 20:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:02:44.547671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dd2b3-2130-4e47-9b02-a186f4c39917 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.553889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 818/1619] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/users => generated 309 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:44.554756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.563693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dd2b3-2130-4e47-9b02-a186f4c39917 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.574111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.574298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa267a-4654-4162-9879-76fe72fa63a2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.575285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.575817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.576337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.577513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.579762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.580295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.580539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.580757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.581034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.639835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 802/1620] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:44.641959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.660098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.662054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0146c30-4a3c-4eb2-a42c-9f8108c37a65 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.676360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 819/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:44.678281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.678658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.679090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.704062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:44.705868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:44.737947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:44.782654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39801a70-68dc-44ab-8902-3b701b36fb20 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.795944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.798163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.798448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.798725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.824751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 820/1622] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:44.825820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.884750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:44.904419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6fee7-193f-4b72-82e8-eeb62c48eaf1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.910712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.911775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.912113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.912799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.913432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.915164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.927128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 821/1623] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:44.927968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:44.941530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.944361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.949057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ec1e-ad28-48bc-85c0-5b4e80459fa5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:44.956009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:44.956713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:44.957563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:44.959751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:44.960514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:44.962964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:44.967298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 803/1624] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:44.969093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.042644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9c17f-6064-46c4-83d2-efb350804fb0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.042644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.042644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.042644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.045736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.046031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 822/1625] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:44 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:45.046031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.053607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583242f6-10f7-4d8d-aa5e-bd434653c55e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.062446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.063305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.063724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.064297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] PATH_INFO: `/v3/projects/4317a7274682482fb694e3721e4ed8e4/users/1e52ff4637b547a4b46f795a4fbf2241/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.086657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4317a7274682482fb694e3721e4ed8e4, user_id=1e52ff4637b547a4b46f795a4fbf2241, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.092163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.125677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 823/1626] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:45 2022] PUT /identity/v3/projects/4317a7274682482fb694e3721e4ed8e4/users/1e52ff4637b547a4b46f795a4fbf2241/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:45.127278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.161828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.166025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d0dd8-1478-41fc-b63f-a06d284e3704 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.181358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 804/1627] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:45.183532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c1af-7229-4e47-a495-b5332898128c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.183726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c1af-7229-4e47-a495-b5332898128c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.184012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c1af-7229-4e47-a495-b5332898128c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.185369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 805/1628] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 17 20:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:02:45.186163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487c1af-7229-4e47-a495-b5332898128c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.217310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce27e1-86e8-4dda-a3c9-ecf9e899830d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.228128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.228811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.229223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.229653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.230470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.232431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9240aa2-fea3-4cff-9da6-da2ccc6b76d6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.255961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 824/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:45.255961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.255961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.255961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.271499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:45.279863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:45.282166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:45.284145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487c1af-7229-4e47-a495-b5332898128c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.293031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.294219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.295265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.296222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.298040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.300946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.319366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 806/1630] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:45.319802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.331294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d6b56-0501-41b6-8957-0fad5db0015a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.336410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.337091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.337376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.337789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] PATH_INFO: `/v3/projects/25ae8a5188fc406fbb72749e8ff82699/users/e8c671f1410f47c7bf7b944ebfdd303b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.356062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ae8a5188fc406fbb72749e8ff82699, user_id=e8c671f1410f47c7bf7b944ebfdd303b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.359458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.384826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 825/1631] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:45.386295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.392944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 807/1632] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:45 2022] PUT /identity/v3/projects/25ae8a5188fc406fbb72749e8ff82699/users/e8c671f1410f47c7bf7b944ebfdd303b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:45.396657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.407347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd370d1-b1ec-4750-a1c9-1163a957830a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.478861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:45.483816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.484548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.484982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.485446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] PATH_INFO: `/v3/projects/4317a7274682482fb694e3721e4ed8e4/users/1e52ff4637b547a4b46f795a4fbf2241/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.488578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4317a7274682482fb694e3721e4ed8e4, user_id=1e52ff4637b547a4b46f795a4fbf2241, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.491758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.500041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142439ab-b469-41e8-a47f-1193ad4de189 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.506664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 826/1633] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:45 2022] PUT /identity/v3/projects/4317a7274682482fb694e3721e4ed8e4/users/1e52ff4637b547a4b46f795a4fbf2241/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:45.509097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.610803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.611659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.612194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.612592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.614586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.630026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af59c16-4905-4811-b3a4-acd25957b2bf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.638848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9965e-425b-47be-8b3c-73152b4a06be admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.643687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 808/1634] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:45.645284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.645762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.646232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.647510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.648142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.648412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.648878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.682170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] MFA Rules not processed for user `1e52ff4637b547a4b46f795a4fbf2241`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:45.683738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:45.745290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:45.788913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.790673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01e583-a399-4b37-8ffe-0e482536983c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.801924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 809/1635] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:45.802772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.808253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 827/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:45.810469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70289db3-e7d0-427a-a0db-445f3e8b0c1b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.810760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70289db3-e7d0-427a-a0db-445f3e8b0c1b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.811038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70289db3-e7d0-427a-a0db-445f3e8b0c1b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.812875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 828/1637] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 17 20:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:02:45.813451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70289db3-e7d0-427a-a0db-445f3e8b0c1b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:45.825910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70289db3-e7d0-427a-a0db-445f3e8b0c1b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.830370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.831427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.831712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.832017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.834270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecd44b-ee41-4168-8557-53bc70986a73 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:45.847316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:45.848398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.848903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.849390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] PATH_INFO: `/v3/projects/25ae8a5188fc406fbb72749e8ff82699/users/e8c671f1410f47c7bf7b944ebfdd303b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.853376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ae8a5188fc406fbb72749e8ff82699, user_id=e8c671f1410f47c7bf7b944ebfdd303b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.855379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:45.856862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa5e51-5737-4cea-982d-b08c34de07cc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.858239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd3ac2-1024-4a20-bb53-4f92a9b40737 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:45.871253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 829/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:45.874570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.883289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.883570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.892568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 810/1639] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:45 2022] PUT /identity/v3/projects/25ae8a5188fc406fbb72749e8ff82699/users/e8c671f1410f47c7bf7b944ebfdd303b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:45.907837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:45.908153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:45.908376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:45.944773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:45.947983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:45.982775 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] MFA Rules not processed for user `e8c671f1410f47c7bf7b944ebfdd303b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:45.984208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:46.092764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:46.112206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 830/1640] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:46.134209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 811/1641] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:46.171065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:46.176277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:46.304242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87c29f-c512-483e-a872-175fd44f10aa None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:46.309919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:46.311101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:46.311616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:46.312114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:46.324957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:46.327294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:46.337333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7bc56-a4e1-49c6-92b7-df7ee7d8af36 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:46.349790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:46.350593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:46.350944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:46.352197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] PATH_INFO: `/v3/users/15a2ed7e5b124aa5ba934b91e1a66afd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:46.354936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=15a2ed7e5b124aa5ba934b91e1a66afd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:46.355500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 812/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:46.358361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:46.454256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:46.472117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f2882-a593-49e3-96e8-f4a9bd9b1a72 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:46.487620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:46.490146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15a2ed7e5b124aa5ba934b91e1a66afd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:46.493360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:46.493682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:46.493960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:46.519264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:46.526762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15a2ed7e5b124aa5ba934b91e1a66afd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:46.532456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:46.535555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a534289-7230-4306-8da7-bebe858937b1 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:46.563558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 813/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:02:46.576841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15a2ed7e5b124aa5ba934b91e1a66afd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:46.598644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15a2ed7e5b124aa5ba934b91e1a66afd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:46.599411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15a2ed7e5b124aa5ba934b91e1a66afd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:46.612884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 831/1644] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:02:46 2022] DELETE /identity/v3/users/15a2ed7e5b124aa5ba934b91e1a66afd => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:02:46.617241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:46.617519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:46.617980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:46.652315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:46.655346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:46.804628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 814/1645] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:02:46.849182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:46.850475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:46.984691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd7cd-9e81-45c0-8f97-77d6c6c4a4cc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:46.989921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:46.990736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:46.991016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:46.991436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:47.003250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:47.004826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:47.017857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 832/1646] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:47.076066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:47.080283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:47.081382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:47.082267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:47.105774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f7a4c-9c0e-4387-9d94-29734e63b402 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:47.116586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:47.117353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:47.118492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:47.119063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] PATH_INFO: `/v3/users/115ba18c2f034f9ea698e5a426846175` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:47.120869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] RBAC: Authorizing `identity:delete_user(user_id=115ba18c2f034f9ea698e5a426846175)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:47.122985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:47.244942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115ba18c2f034f9ea698e5a426846175'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:47.265915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115ba18c2f034f9ea698e5a426846175'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:47.302296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b69303-0ffa-4c48-8f1e-a7d311ab9ec4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:47.304297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115ba18c2f034f9ea698e5a426846175'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:47.313245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:47.314455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:47.314809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:47.315242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:47.317735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:47.321229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115ba18c2f034f9ea698e5a426846175'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:47.321993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7449b15c-1044-410e-8072-4fd721be427d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115ba18c2f034f9ea698e5a426846175'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:47.334543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 833/1647] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:02:47 2022] DELETE /identity/v3/users/115ba18c2f034f9ea698e5a426846175 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:47.337599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:47.338247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:47.338615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:47.363297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:47.364939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:47.440297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:47.473683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 834/1648] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:02:47.482306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:47.528090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:47.530862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:47.551935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:47.559139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:47.583063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 815/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:47.633219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bf149-acb0-43e4-bb64-7037c54acc8a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:47.637566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:47.638940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:47.639324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:47.639698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:47.667302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:47.669548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:47.702027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 835/1650] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:48.695206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:48.712070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60db8ba-88e0-4f4f-a59f-ddfd36e5810d service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:48.723514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:48.724550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:48.724796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:48.725074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] PATH_INFO: `/v3/projects/c0c02701e05249459322208ed51226f4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:48.729526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c02701e05249459322208ed51226f4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:48.734223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:48.825507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:48.827576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:48.845818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c02701e05249459322208ed51226f4', 'request_id': 'req-3bfa476a-e818-425d-ae48-92e7ac26293e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:48.848132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e14333-9d5c-432e-b15a-7d1245d78f62 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:48.856607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c02701e05249459322208ed51226f4', 'request_id': 'req-3bfa476a-e818-425d-ae48-92e7ac26293e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:48.880954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c02701e05249459322208ed51226f4', 'request_id': 'req-3bfa476a-e818-425d-ae48-92e7ac26293e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:48.902005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] The token cache is being invalidate because project c0c02701e05249459322208ed51226f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:02:48.902635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:48.903774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bfa476a-e818-425d-ae48-92e7ac26293e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:48.907981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 816/1651] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:02:48 2022] DELETE /identity/v3/projects/c0c02701e05249459322208ed51226f4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:48.914314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31abf2-73c8-4b20-bc19-d9bf645e29ca None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:48.914889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31abf2-73c8-4b20-bc19-d9bf645e29ca None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:48.915288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31abf2-73c8-4b20-bc19-d9bf645e29ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:48.967202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ea31abf2-73c8-4b20-bc19-d9bf645e29ca None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:48.971940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ea31abf2-73c8-4b20-bc19-d9bf645e29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:48.995747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:48.996886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:48.997185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:48.997420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] PATH_INFO: `/v3/projects/10570b24f117432d9b887de535ea8764` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:48.999162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=10570b24f117432d9b887de535ea8764)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:49.000728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: WARNING py.warnings [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:49.000728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: warnings.warn(msg) Aug 17 20:02:49.000728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: Aug 17 20:02:49.002328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:49.139274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 817/1652] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:02:49.142997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:49.143563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:49.144016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:49.149981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10570b24f117432d9b887de535ea8764', 'request_id': 'req-be53df70-fd2b-431b-bb16-4b14cdb534cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.158655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10570b24f117432d9b887de535ea8764', 'request_id': 'req-be53df70-fd2b-431b-bb16-4b14cdb534cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.177062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10570b24f117432d9b887de535ea8764', 'request_id': 'req-be53df70-fd2b-431b-bb16-4b14cdb534cf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.196582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:49.197875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:49.220450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] The token cache is being invalidate because project 10570b24f117432d9b887de535ea8764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:02:49.220746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.222132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.224550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 836/1653] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:02:48 2022] DELETE /identity/v3/projects/10570b24f117432d9b887de535ea8764 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:49.233267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:49.237088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:49.302082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:49.341019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:49.374400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 818/1654] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:49.378600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:49.399345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df64e79-20f1-48da-83b5-f62ff726a1b6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:49.412735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:49.413361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:49.414336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53df70-fd2b-431b-bb16-4b14cdb534cf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:49.425736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:49.427903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:49.428213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:49.429152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:49.429434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:49.429733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] PATH_INFO: `/v3/users/1e52ff4637b547a4b46f795a4fbf2241` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:49.431265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:49.431601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:49.433213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e52ff4637b547a4b46f795a4fbf2241)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:49.435351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:49.436857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:49.484256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 819/1655] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:02:49.485402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:49.556010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec2356-9c4f-4c35-8211-4b0652435d5d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:49.567957 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e52ff4637b547a4b46f795a4fbf2241'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.587523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:49.587665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e52ff4637b547a4b46f795a4fbf2241'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.588421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:49.588830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:49.589274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:49.590243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:49.593468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:49.637899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e52ff4637b547a4b46f795a4fbf2241'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.687551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e52ff4637b547a4b46f795a4fbf2241'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.688301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e52ff4637b547a4b46f795a4fbf2241'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:49.699111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 837/1656] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:02:49 2022] DELETE /identity/v3/users/1e52ff4637b547a4b46f795a4fbf2241 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:49.717724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 820/1657] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:49 2022] POST /identity/v3/projects => generated 367 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:49.719014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:49.721258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:49.881666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b4234-1be5-4bec-9708-6467099a4e1d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.006574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:50.036668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.038290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.041136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.041298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.042045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.047830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:50.117670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 838/1658] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:49 2022] POST /identity/v3/users => generated 284 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:50.122077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:50.141272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a6f87-6692-45c3-b64f-0763de8c6623 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.159014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.159720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.160006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.160283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.163233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.166346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:50.190964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 821/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:02:50.196318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:50.211427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383fd6-7492-4ce0-a444-8823ea26277b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.220791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.221918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.222994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.224574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.226626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.229258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82bb492-ff1e-43b8-88ff-94079a691b0e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:50.248951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 839/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:50.252296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:50.267087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a9569-648a-4db5-88e9-451fbdfba2b5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.274584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.276654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.277594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.278050 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] PATH_INFO: `/v3/projects/95d1b8374d7c477eb89d67ae84532aaf/users/865d81195b6c4611917ef321bbab6455/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.318500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d1b8374d7c477eb89d67ae84532aaf, user_id=865d81195b6c4611917ef321bbab6455, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.321233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:50.370136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 822/1661] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:50 2022] PUT /identity/v3/projects/95d1b8374d7c477eb89d67ae84532aaf/users/865d81195b6c4611917ef321bbab6455/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:50.374907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.375270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.375593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.461626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] MFA Rules not processed for user `865d81195b6c4611917ef321bbab6455`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:50.463213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:50.623597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 840/1662] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:50.626585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:50.763177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f573a0-1283-4c19-88c5-138742ef0fae admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.775357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.778485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.778833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.779113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.779835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.789485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:50.842065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 823/1663] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:50 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:50.842382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:50.856431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a1591-3445-4768-8a74-36b0c40ece86 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:50.983819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:50.984623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:50.984990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:50.985330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:50.986413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:50.988548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.045038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 841/1664] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:50 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:02:51.049337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.069077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05bbd1-bdc8-4ead-a12a-2d02f7f42343 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.084851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.085981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.085981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.086215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.087221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.092821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.118672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 824/1665] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:51.122729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.138912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8c405-1ab7-4ba3-947e-2aa195d537bf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.149708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.150686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.151149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.151806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] PATH_INFO: `/v3/projects/aedef472a644454e8c8d86536aa9d63c/users/3e151f29c601461ea78267225c220ea1/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.232865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedef472a644454e8c8d86536aa9d63c, user_id=3e151f29c601461ea78267225c220ea1, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.237002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.275932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 842/1666] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:51 2022] PUT /identity/v3/projects/aedef472a644454e8c8d86536aa9d63c/users/3e151f29c601461ea78267225c220ea1/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:51.280479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.283149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.287223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:51.329953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c9520-6755-4927-8acd-e6e0f665e6f0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.385155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:51.542575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c8a2f-9065-4d20-8d26-b8bed3715afd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.543347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:51.561608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.564978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.565168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.565406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.566416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.570825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.589573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.589573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.589573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.589573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] PATH_INFO: `/v3/users/20968c5893fe42e79547c978d6873845` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.590215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] RBAC: Authorizing `identity:delete_user(user_id=20968c5893fe42e79547c978d6873845)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.590215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: WARNING py.warnings [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 17 20:02:51.590215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: warnings.warn(msg) Aug 17 20:02:51.590215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: Aug 17 20:02:51.590215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.603506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 843/1667] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:51.604388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.619789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:51.651529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080729d3-e49c-4fda-9780-6ff69e16b07b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.669440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.673235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.673235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.675810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] PATH_INFO: `/v3/projects/4317a7274682482fb694e3721e4ed8e4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.680396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4317a7274682482fb694e3721e4ed8e4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.685092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.699915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20968c5893fe42e79547c978d6873845'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.700827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20968c5893fe42e79547c978d6873845'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.702808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20968c5893fe42e79547c978d6873845'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.741455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20968c5893fe42e79547c978d6873845'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.765698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20968c5893fe42e79547c978d6873845'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.784334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4317a7274682482fb694e3721e4ed8e4', 'request_id': 'req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.794289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 825/1668] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:02:51 2022] DELETE /identity/v3/users/20968c5893fe42e79547c978d6873845 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:51.795410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:51.810534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:51.812332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4317a7274682482fb694e3721e4ed8e4', 'request_id': 'req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.831980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4317a7274682482fb694e3721e4ed8e4', 'request_id': 'req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.862756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] The token cache is being invalidate because project 4317a7274682482fb694e3721e4ed8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:02:51.863050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.864559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-10ac8e78-8fa8-4fd3-a0ed-bcf30a1fc9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:51.867343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 844/1669] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:02:51 2022] DELETE /identity/v3/projects/4317a7274682482fb694e3721e4ed8e4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:51.868690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ceb8-ea46-4f45-aa11-4b18991c1fe3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.870749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ceb8-ea46-4f45-aa11-4b18991c1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.871072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566ceb8-ea46-4f45-aa11-4b18991c1fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.915425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5566ceb8-ea46-4f45-aa11-4b18991c1fe3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:51.918229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5566ceb8-ea46-4f45-aa11-4b18991c1fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:51.960463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f3bf6-8688-4cc1-b22e-5af40b71efac admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:51.970494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:51.971522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.972009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.972476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] PATH_INFO: `/v3/projects/aedef472a644454e8c8d86536aa9d63c/users/3e151f29c601461ea78267225c220ea1/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:51.975443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedef472a644454e8c8d86536aa9d63c, user_id=3e151f29c601461ea78267225c220ea1, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:51.978729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407829c0-3b86-4b92-bc42-eb5b350f2cc2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:51.996157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 826/1670] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:51 2022] PUT /identity/v3/projects/aedef472a644454e8c8d86536aa9d63c/users/3e151f29c601461ea78267225c220ea1/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:51.997887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:51.998397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:51.998873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.049154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] MFA Rules not processed for user `454c2bf24ed94f82b453df36e6e8eac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:52.050688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.052325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 845/1671] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:52.053359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d1a4-7d59-48fb-a947-3f0f2617adb9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.053670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d1a4-7d59-48fb-a947-3f0f2617adb9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.053942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645d1a4-7d59-48fb-a947-3f0f2617adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.123944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f645d1a4-7d59-48fb-a947-3f0f2617adb9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:52.125110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f645d1a4-7d59-48fb-a947-3f0f2617adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.191536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:52.247682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 846/1672] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:52.249181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4930e3-e800-4565-a0ca-6ef42d3bbb18 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.249459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4930e3-e800-4565-a0ca-6ef42d3bbb18 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.249758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4930e3-e800-4565-a0ca-6ef42d3bbb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.277680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:52.283640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 827/1673] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:52.286736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:52.289409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.312308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7f4930e3-e800-4565-a0ca-6ef42d3bbb18 None None] MFA Rules not processed for user `3e151f29c601461ea78267225c220ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:52.314558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7f4930e3-e800-4565-a0ca-6ef42d3bbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.443400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b19487-5020-48d8-b53a-f6be4bd04803 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:52.452147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:52.452883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.453163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.453432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.477616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:52.479063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:52.502522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 847/1674] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:52.503838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.504122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.504392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.507464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 828/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:52.510332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:52.512548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.543022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:52.544258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.567132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 848/1676] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:52.571328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:52.573358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:52.686488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd536cc-a0ef-4232-bf3a-16c80ce20e2c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:52.694247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:52.695156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.696257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.696661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.706401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4735-2084-4759-a97b-f0a7c50dbbb7 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:52.723913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:52.725702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:52.726771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:52.728474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.728900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.729303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.741078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:52.743051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:52.755611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 849/1677] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:52.756497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:52.759307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 829/1678] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:52.768707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5642e3a-3ec9-4525-a57b-4d7e40bc8e85 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:52.782004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:52.782546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.782816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.783081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.784105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:52.784406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:52.786085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:52.819275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 850/1679] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:52.820850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:52.854458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4bf43-e93b-4a28-a12f-807ebd6b929b service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:52.872465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:52.875357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:52.875667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:52.876837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:52.877787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:52.892977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:52.996421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 830/1680] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:52 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:53.000480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:53.019209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f9f3a-f20a-414d-9081-9fba0f3cfe96 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:53.021281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:53.204107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:53.235552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970121d-239a-4fc5-8934-17e126f2220d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:53.246132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:53.246853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.247128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.247397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.253180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:53.283011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:53.284012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.284012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.284183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.284812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:53.293975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:53.306778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:53.365588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 851/1681] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:52 2022] POST /identity/v3/users => generated 290 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:53.369397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:53.384834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92234fd-ced9-46aa-86f3-4689d4a1b426 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:53.390787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:53.393039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.395729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.397805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.403368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:53.407789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:53.418430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:53.423644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:53.435580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 852/1682] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:53.439682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:53.447040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 831/1683] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:02:53.461280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809bcb-a299-4f77-8aab-78a0406c9349 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:53.472529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:53.474160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.474450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.474727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.478098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:53.479855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ccd28-ca64-4070-bf56-083ab8c974a3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:53.508988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 853/1684] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:53.513962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:53.534287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1175f-7d6c-40dc-802b-578865513547 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:53.549368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:53.550223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.550751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.551138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] PATH_INFO: `/v3/projects/082124a3989c4e928bbda32cceb7d286/users/23fb3bb0bc1441158dc2ffac03c0d22f/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.593165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=082124a3989c4e928bbda32cceb7d286, user_id=23fb3bb0bc1441158dc2ffac03c0d22f, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:53.599435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:53.627312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 832/1685] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:53 2022] PUT /identity/v3/projects/082124a3989c4e928bbda32cceb7d286/users/23fb3bb0bc1441158dc2ffac03c0d22f/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:53.638931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:53.639242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:53.639488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:53.697494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] MFA Rules not processed for user `23fb3bb0bc1441158dc2ffac03c0d22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:53.701405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:53.880691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 854/1686] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:53.885829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.044023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee7a73-df47-4a66-b9d8-d96299b4b500 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.054461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.056000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.056293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.056569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.059260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.066272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.127796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 833/1687] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:53 2022] POST /identity/v3/projects => generated 381 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:54.131896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.148481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e8e38-84bf-4775-8a0e-c6848e4e6ddf None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.283750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.285300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.285813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.286300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.287302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.289518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.339644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 855/1688] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:54 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:54.343572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.360171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577510a2-6dc0-4ff6-997e-a43f64c77a7c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.371749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.373648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.373941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.374577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.377068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.379880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.415556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 834/1689] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:54.419816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.432653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf018474-97ff-49bd-a4d0-aac315480992 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.438224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.441951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.442492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.443267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] PATH_INFO: `/v3/projects/0bd1897121bd4c02a01d4f8487d7623e/users/4a962757b4d54c7685fff119ecdedbcd/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.475679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd1897121bd4c02a01d4f8487d7623e, user_id=4a962757b4d54c7685fff119ecdedbcd, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.478994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.503683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 856/1690] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:54 2022] PUT /identity/v3/projects/0bd1897121bd4c02a01d4f8487d7623e/users/4a962757b4d54c7685fff119ecdedbcd/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:54.507765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.519387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4c4ea-d034-472e-b6c3-a3b24df8e0a4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.619475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.620287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.620729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.621066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.621957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.624128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb270c07-cd4d-4aac-89eb-e7fb88b9bc90 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.640949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 835/1691] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:54.645831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.659106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74cd21-af25-4807-80e9-099d3eecec00 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.664966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.664966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.665275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.665528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] PATH_INFO: `/v3/projects/0bd1897121bd4c02a01d4f8487d7623e/users/4a962757b4d54c7685fff119ecdedbcd/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.668046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd1897121bd4c02a01d4f8487d7623e, user_id=4a962757b4d54c7685fff119ecdedbcd, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.671816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.679604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:54.697668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 857/1692] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:54 2022] PUT /identity/v3/projects/0bd1897121bd4c02a01d4f8487d7623e/users/4a962757b4d54c7685fff119ecdedbcd/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:54.697908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.698155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.698485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.720928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:54.745157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.747045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:54.756768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] MFA Rules not processed for user `4a962757b4d54c7685fff119ecdedbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:54.758817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:54.855083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71139403-801a-4d13-b08e-82fec18eab6d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:54.878855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:54.882165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:54.883460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:54.884166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] PATH_INFO: `/v3/projects/83184ecf125d4a99894aad39a9046b12` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:54.892027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83184ecf125d4a99894aad39a9046b12)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:54.897079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 836/1693] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:54.900091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:54.910086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:54.911696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.015249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83184ecf125d4a99894aad39a9046b12', 'request_id': 'req-a93add04-12a3-4b2e-a8d9-da53f8ff6370'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:55.030861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83184ecf125d4a99894aad39a9046b12', 'request_id': 'req-a93add04-12a3-4b2e-a8d9-da53f8ff6370'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:55.047859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83184ecf125d4a99894aad39a9046b12', 'request_id': 'req-a93add04-12a3-4b2e-a8d9-da53f8ff6370'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:55.078851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085a6f1-5723-482f-b509-64a92d04c5c2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.082241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] The token cache is being invalidate because project 83184ecf125d4a99894aad39a9046b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:02:55.082991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:55.084386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a93add04-12a3-4b2e-a8d9-da53f8ff6370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:55.093401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 858/1694] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:02:54 2022] DELETE /identity/v3/projects/83184ecf125d4a99894aad39a9046b12 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:55.097092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.099506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.099805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.186413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:55.194494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.288647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.289317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.289655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.289999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.294268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.394350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.395932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931f112-5815-4745-9dc8-b19405a62bc6 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.400526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:55.423850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 837/1695] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:02:55.437518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 859/1696] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:55.440620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.441111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.441617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.487598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:55.488912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.501412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 838/1697] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:55.506123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.523279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298d8a0-13fe-4729-9037-60f5e028f5db None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.532811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.533475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.534201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.534201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.534740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:02:55.535045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.536967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.564828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 860/1698] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:02:55.575177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.584574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.586427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.587838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79515af-1ab1-4caa-87b0-e55c70401e80 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.592920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.593701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.594052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.594788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.595512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.597637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.608806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45400f7e-cc81-4bdf-8701-5e622424a6c3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.618302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.618988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.619295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.619593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.626611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:55.637608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 839/1699] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:55 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:55.642367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.654818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:55.752409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.753929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.779286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 861/1700] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:02:55.795677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0897f8c2-49fb-43a3-a88c-5f0b5da0a6b4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.814072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.818015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.818240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.818526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.819209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.827969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.902586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 840/1701] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:55 2022] POST /identity/v3/users => generated 294 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:55.903619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.920536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8be67-0d20-4fa3-9a9e-5fd401107a73 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:55.936449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:55.937090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:55.937379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:55.937694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:55.938319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:55.939973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:55.972272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 862/1702] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:55.973850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:55.991791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f111c9-6ebb-44c5-8a0d-faa11db7f77a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.005831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.006497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.006786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.007063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.007734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:56.009393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05bcc9-e172-417d-9220-a88214ddbdce admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:56.038751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 841/1703] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:56.040051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:56.063942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045b0ac-ede1-4e5d-830c-76f7593baa03 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.090100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.091120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.091601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.092151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] PATH_INFO: `/v3/projects/77eeae00343b41b9925bf6f1eea5d7f7/users/4714bb780b9a4f1d85151161b30a4566/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.144006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77eeae00343b41b9925bf6f1eea5d7f7, user_id=4714bb780b9a4f1d85151161b30a4566, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:56.147609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:56.173019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 863/1704] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:56 2022] PUT /identity/v3/projects/77eeae00343b41b9925bf6f1eea5d7f7/users/4714bb780b9a4f1d85151161b30a4566/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:56.183664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.183958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.184233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.264052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] MFA Rules not processed for user `4714bb780b9a4f1d85151161b30a4566`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:56.266438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:56.418191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 842/1705] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:56.419381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:56.586440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4ca1c-91cc-473b-9c96-a2499b1c1130 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.604726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.605398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.605760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.606037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.606689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:56.609328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:56.664808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 864/1706] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:02:56 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:56.672503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:56.691986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4ea93-74ed-4959-ba72-ab65aae8e756 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.741738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:56.795986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.797061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.797430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.797791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.798313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:56.805925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:56.858597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 843/1707] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:02:56 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:02:56.862774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:56.879583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742728ee-5a87-4946-92cb-9e72dd129e6b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.891001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.892026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.892288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.893882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:56.895719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:56.899970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:56.928634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 865/1708] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:56.933134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:56.946772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7464b06-d023-426b-8ee0-4c0062900039 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:56.959120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:56.959831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:56.960609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:56.960781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] PATH_INFO: `/v3/projects/399c37b2bd484a069779de068f901cfc/users/c395e5bcdbd842fdb4e93f988dd6c3bf/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:57.000234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399c37b2bd484a069779de068f901cfc, user_id=c395e5bcdbd842fdb4e93f988dd6c3bf, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:57.005865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:57.038032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 844/1709] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:56 2022] PUT /identity/v3/projects/399c37b2bd484a069779de068f901cfc/users/c395e5bcdbd842fdb4e93f988dd6c3bf/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:57.042103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:57.059863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b89178-82cd-4168-82b1-27cd6914b15e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:57.129470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:57.164956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:57.169481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:57.170203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:57.170851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:57.174656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:57.179291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd38ce-8a95-4329-8cbd-33499bb34cb3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:57.211134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:57.217826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 866/1710] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:02:57.244714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303883c-ae17-46c8-ba52-39db6c717212 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:57.258256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:57.259305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:57.260041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:57.261456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] PATH_INFO: `/v3/projects/399c37b2bd484a069779de068f901cfc/users/c395e5bcdbd842fdb4e93f988dd6c3bf/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:57.267257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=399c37b2bd484a069779de068f901cfc, user_id=c395e5bcdbd842fdb4e93f988dd6c3bf, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:57.270171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:57.312569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 845/1711] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:02:57 2022] PUT /identity/v3/projects/399c37b2bd484a069779de068f901cfc/users/c395e5bcdbd842fdb4e93f988dd6c3bf/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:57.324699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:57.324972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:57.325245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:57.362529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] MFA Rules not processed for user `c395e5bcdbd842fdb4e93f988dd6c3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:57.364799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:57.557712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 867/1712] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:57.579899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:57.582352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:57.656945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd076-5527-42b5-84e3-cb932227e3ca admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:57.661874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:57.662748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:57.663215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:57.663784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:57.675321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:57.677178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fa346-0be1-43a9-b4a0-bb58ef730c5c service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:57.688499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 846/1713] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:02:57.811800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:58.001256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bea89-4b80-4e70-8ad7-fe20b557fa3f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:58.017590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:58.018506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:58.018946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:58.022365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] PATH_INFO: `/v3/users/4714bb780b9a4f1d85151161b30a4566` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:58.024883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4714bb780b9a4f1d85151161b30a4566)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:58.033587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:58.159248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4714bb780b9a4f1d85151161b30a4566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:58.180755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4714bb780b9a4f1d85151161b30a4566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:58.201824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4714bb780b9a4f1d85151161b30a4566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:58.207673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:58.219285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4714bb780b9a4f1d85151161b30a4566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:58.220225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b6e0ba5-f781-43f8-b386-e4e9462ee76e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4714bb780b9a4f1d85151161b30a4566'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:02:58.240780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 868/1714] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:02:57 2022] DELETE /identity/v3/users/4714bb780b9a4f1d85151161b30a4566 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:02:58.245878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374140-0007-4af1-beaa-b7e162379693 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:58.246341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374140-0007-4af1-beaa-b7e162379693 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:58.246481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374140-0007-4af1-beaa-b7e162379693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:58.292111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-99374140-0007-4af1-beaa-b7e162379693 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:58.295944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-99374140-0007-4af1-beaa-b7e162379693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:58.496024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 847/1715] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:58.553816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09086-00b4-4b76-8fc2-b80889cb3802 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:58.554384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09086-00b4-4b76-8fc2-b80889cb3802 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:58.554793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec09086-00b4-4b76-8fc2-b80889cb3802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:58.583611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99374140-0007-4af1-beaa-b7e162379693 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:02:58.589891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-99374140-0007-4af1-beaa-b7e162379693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:58.594385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4ec09086-00b4-4b76-8fc2-b80889cb3802 None None] MFA Rules not processed for user `454c2bf24ed94f82b453df36e6e8eac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:02:58.598432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4ec09086-00b4-4b76-8fc2-b80889cb3802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:02:58.706209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:02:58.739189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 869/1716] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:02:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:02:58.745329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99374140-0007-4af1-beaa-b7e162379693 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:02:58.749445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:02:58.750374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:02:58.751396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:02:58.751875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:02:58.762596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:02:58.764643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ac659-5b8c-46e6-9ecf-7a86d6f3c805 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:02:58.777168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 848/1717] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:00.103128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:00.104154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:00.104647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:00.106213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 870/1718] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 17 20:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:00.117429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479ce9-6f4b-4736-bcbb-a26733f89103 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:00.118584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479ce9-6f4b-4736-bcbb-a26733f89103 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:00.119008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479ce9-6f4b-4736-bcbb-a26733f89103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:00.220994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8e479ce9-6f4b-4736-bcbb-a26733f89103 None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:00.222284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e479ce9-6f4b-4736-bcbb-a26733f89103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:00.260902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 849/1719] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:03:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:00.274888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:00.277228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:00.316579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824adc9f-c7da-47e9-8653-ce23bf373002 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:00.325851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:00.328895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:00.329889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:00.330430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:00.349347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:00.350916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc632a-08c3-41ae-914c-14f4013e655a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:00.422017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 871/1720] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:00.603526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:00.604044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:00.604469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:00.668331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] MFA Rules not processed for user `b095ab3c72b14c67adf61ab4189e4892`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:00.669834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:00.811473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 850/1721] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:00.824457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a06f2-f881-4f9f-bb3f-28fce405f46a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:00.824853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a06f2-f881-4f9f-bb3f-28fce405f46a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:00.825136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a06f2-f881-4f9f-bb3f-28fce405f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:00.888090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-235a06f2-f881-4f9f-bb3f-28fce405f46a None None] MFA Rules not processed for user `b095ab3c72b14c67adf61ab4189e4892`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:00.889774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-235a06f2-f881-4f9f-bb3f-28fce405f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:00.920494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 872/1722] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:00.921939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:00.923617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:01.064189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.064358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.064693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.099212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b72b84-48ff-49bf-bb59-c6fd3f15945f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:01.115583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:01.116229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.116504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.116885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.139197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:01.140103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] MFA Rules not processed for user `a5aece8a7f2342cfabd9a3f8707a4470`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:01.141390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:01.142171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e5e3b-7a90-435a-ad6d-88ceab7048b9 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:01.177306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 851/1723] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:01.392264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 873/1724] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:01.402498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032586-e367-43e5-a2be-9a566c021b9d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.402780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032586-e367-43e5-a2be-9a566c021b9d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.403061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032586-e367-43e5-a2be-9a566c021b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.474557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-35032586-e367-43e5-a2be-9a566c021b9d None None] MFA Rules not processed for user `a5aece8a7f2342cfabd9a3f8707a4470`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:01.475682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-35032586-e367-43e5-a2be-9a566c021b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:01.515457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 852/1725] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:01.578271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:01.582733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:01.612772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729b57-b52e-45c0-8121-49f9c06a5128 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:01.620186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:01.621105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.621738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.622205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] PATH_INFO: `/v3/projects/77eeae00343b41b9925bf6f1eea5d7f7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.623605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77eeae00343b41b9925bf6f1eea5d7f7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:01.628193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:01.658788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4333e5-386b-450f-9a18-ae2c811fe682 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.658788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4333e5-386b-450f-9a18-ae2c811fe682 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.659032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4333e5-386b-450f-9a18-ae2c811fe682 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.662302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 853/1726] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 17 20:03:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:01.678301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:01.678584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:01.679007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:01.738988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:01.740618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:01.743854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77eeae00343b41b9925bf6f1eea5d7f7', 'request_id': 'req-5a798cf3-649a-4c76-9355-ea264a2f6751'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:01.799466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77eeae00343b41b9925bf6f1eea5d7f7', 'request_id': 'req-5a798cf3-649a-4c76-9355-ea264a2f6751'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:01.814368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:01.831492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77eeae00343b41b9925bf6f1eea5d7f7', 'request_id': 'req-5a798cf3-649a-4c76-9355-ea264a2f6751'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:01.838668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:01.875652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] The token cache is being invalidate because project 77eeae00343b41b9925bf6f1eea5d7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:01.879242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:01.881354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:01.895061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 874/1727] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:01 2022] DELETE /identity/v3/projects/77eeae00343b41b9925bf6f1eea5d7f7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:01.896104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:01.900322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.004862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 854/1728] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 17 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:02.040911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:02.069369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423394ba-5763-43b4-bb86-aa0f6094a7f1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:02.133038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:02.133718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.136280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.136567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.147086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.255045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:02.257967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:02.273827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 855/1729] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:02.273964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a798cf3-649a-4c76-9355-ea264a2f6751 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:02.275537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:02.277327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.299314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:02.299997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.300287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.300552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] PATH_INFO: `/v3/users/c395e5bcdbd842fdb4e93f988dd6c3bf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.304091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c395e5bcdbd842fdb4e93f988dd6c3bf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:02.309072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:02.390817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d9b94-779e-488f-b048-1d88a529c065 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:02.403528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:02.404387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.404845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.405213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.407940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.509004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c395e5bcdbd842fdb4e93f988dd6c3bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:02.526064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c395e5bcdbd842fdb4e93f988dd6c3bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:02.590269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c395e5bcdbd842fdb4e93f988dd6c3bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:02.606850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c395e5bcdbd842fdb4e93f988dd6c3bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:02.607487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3cbd5a1f-6305-4969-a911-24a95165379c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c395e5bcdbd842fdb4e93f988dd6c3bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:02.616070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 875/1730] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:01 2022] DELETE /identity/v3/users/c395e5bcdbd842fdb4e93f988dd6c3bf => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:02.617521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab7d1-e095-402d-8e75-1d701301e4b8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.617838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab7d1-e095-402d-8e75-1d701301e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.618112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eab7d1-e095-402d-8e75-1d701301e4b8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.624757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 876/1731] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 17 20:03:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:02.626026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:02.626142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.626414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.626685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.627902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b692d0-b52d-49fd-abab-b060fc8d767d service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:02.654225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 856/1732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:02.655693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:02.656119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:02.656515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:02.677407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] MFA Rules not processed for user `3a40f55377b64f6f93784eefaf28152e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:02.678730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.699468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:02.700833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:02.841661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:02.853154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 877/1733] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:02.896487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:02.936987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 857/1734] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:03:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:03.049905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:03.051615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:03.082562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be7513-718f-4862-95c6-673fb22d4778 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:03.087266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:03.088539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:03.088965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:03.089406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:03.099953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:03.104481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:03.126117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 878/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:03.643151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:03.645277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:03.833177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408834dc-6e84-49c7-b1df-3160f8e6c0bb None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:03.842856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:03.842856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:03.848518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:03.848806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] PATH_INFO: `/v3/projects/399c37b2bd484a069779de068f901cfc` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:03.854030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399c37b2bd484a069779de068f901cfc)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:03.857893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:03.948913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399c37b2bd484a069779de068f901cfc', 'request_id': 'req-abc5b635-fc5e-4f19-8e31-742921f98be0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:03.957861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399c37b2bd484a069779de068f901cfc', 'request_id': 'req-abc5b635-fc5e-4f19-8e31-742921f98be0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:03.970028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399c37b2bd484a069779de068f901cfc', 'request_id': 'req-abc5b635-fc5e-4f19-8e31-742921f98be0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:03.988672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:03.989841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] The token cache is being invalidate because project 399c37b2bd484a069779de068f901cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:03.990180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:03.991421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:03.992750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-abc5b635-fc5e-4f19-8e31-742921f98be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:03.998072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 858/1736] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:03 2022] DELETE /identity/v3/projects/399c37b2bd484a069779de068f901cfc => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:04.000311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899d4c-784c-43d4-b0b5-29096544e710 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.000640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899d4c-784c-43d4-b0b5-29096544e710 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.001000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad899d4c-784c-43d4-b0b5-29096544e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.049127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ad899d4c-784c-43d4-b0b5-29096544e710 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:04.052108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad899d4c-784c-43d4-b0b5-29096544e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:04.173642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc20f4e-c4af-4965-b57d-ccb4a7bd5141 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.297688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 859/1737] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:04.312019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.312559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.312977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.392014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:04.393242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:04.424888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 860/1738] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:04.426258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.444137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fcc01-2dd7-4294-a6a9-d93fb7c98fd6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.456574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.457315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.457617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.457891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.458660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:04.458963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.460591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.518545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 861/1739] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:04.519364 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.519486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.520081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.520359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.520622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.535272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:04.542646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:04.549365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1bdfa-c0d4-483f-93ef-27276e858447 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.578535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.579178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.579461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.579826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.580375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.582482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.629731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 862/1740] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:04 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:04.634793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.755608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4833-2c32-4274-be36-49700fa552a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.758774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.766376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.767798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.768579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.769728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.771029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.772828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.773500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.797445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 879/1741] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 809 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:04.802687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 863/1742] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:04 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:04.806922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.822616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd317f8-427d-4cb8-9186-d74d03ead89a service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.831345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.832478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.833120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.833491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.834323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.836363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.850355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 880/1743] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:04.854371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.874740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d0cfe-9540-427e-be61-c964bc4717a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.882876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.883613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.883993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.884521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.885395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:04.887743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebb2b7-61c6-41c6-af34-2208c8630351 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:04.901589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 864/1744] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:04.907048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:04.939011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef9479-cdde-43b1-91d5-e147d93853e9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:04.959932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:04.961108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:04.961411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:04.961718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] PATH_INFO: `/v3/projects/3ff94224f4e44f1ba85a37b12635e304/users/9bc951f09d39481ab3afd0f392207af4/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:04.992868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:05.013796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff94224f4e44f1ba85a37b12635e304, user_id=9bc951f09d39481ab3afd0f392207af4, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:05.020202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9684bd-610e-4103-a600-3a206cd701b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:05.086695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 881/1745] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:04 2022] PUT /identity/v3/projects/3ff94224f4e44f1ba85a37b12635e304/users/9bc951f09d39481ab3afd0f392207af4/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:05.089328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.089328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.089328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.153116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] MFA Rules not processed for user `9bc951f09d39481ab3afd0f392207af4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:05.156944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:05.202840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.203118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.203386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.255179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] MFA Rules not processed for user `a5aece8a7f2342cfabd9a3f8707a4470`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:05.259502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:05.408230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 882/1746] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:05.434184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 865/1747] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:05.439140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:05.564759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80313a-4096-4f82-bacc-6f68c450e2d5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:05.576154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:05.576831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.577105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.577369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.578090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:05.581623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:05.642223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 883/1748] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:05 2022] POST /identity/v3/projects => generated 389 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:05.644146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:05.670007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06990b0a-e6dd-4915-b178-7ffc87ef6a55 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:05.752793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:05.767400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:05.769273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.769767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.770992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.772582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:05.775809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:05.817890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 866/1749] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:05 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:05.821773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:05.834152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5ebf2-826a-42ae-8883-ac2bc7bc5693 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:05.840454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:05.841140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.841415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.841757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.842445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:05.844158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:05.856752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 884/1750] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:05.860955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:05.880712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb10ff-672a-4bb0-8452-406189848da7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:05.887667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:05.888927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:05.889369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:05.889674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] PATH_INFO: `/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1/users/e8d730881eab425db648ab854ebaa639/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:05.911609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0b5436b00c46b98b67fd43dd3289b1, user_id=e8d730881eab425db648ab854ebaa639, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:05.916115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:05.941823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 867/1751] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:05 2022] PUT /identity/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1/users/e8d730881eab425db648ab854ebaa639/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:05.943719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:05.968333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7dec0-cb0b-4970-91ca-f2a686d7f39a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:06.067995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:06.068653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:06.068933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:06.069239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:06.069937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:06.073818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7afe19-c7fd-40df-84ef-59cba1705899 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:06.104174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 885/1752] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:06.107722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:06.124796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b417e-b0ad-4ff8-ac6e-4a01c547bd1c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:06.137891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:06.139347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:06.139616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:06.139890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] PATH_INFO: `/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1/users/e8d730881eab425db648ab854ebaa639/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:06.145696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a0b5436b00c46b98b67fd43dd3289b1, user_id=e8d730881eab425db648ab854ebaa639, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:06.167825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:06.201067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 868/1753] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:06 2022] PUT /identity/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1/users/e8d730881eab425db648ab854ebaa639/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:06.212327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:06.212643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:06.212943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:06.225397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:06.298733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] MFA Rules not processed for user `e8d730881eab425db648ab854ebaa639`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:06.300096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:06.510381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 886/1754] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:06 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:06.531833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:06.535477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:06.685343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f26605-fe59-45a0-8829-90c0e5f1d592 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:06.693583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:06.695086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:06.696669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:06.697215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:06.733583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:06.733583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c587561-5bd9-4e21-a28b-98ddf183ead3 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:06.760620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 869/1755] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:06 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:06.988875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:07.091576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eb734-7263-40e6-a495-f60494bf4afa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:07.100365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:07.101048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:07.101326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:07.104980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] PATH_INFO: `/v3/users/9bc951f09d39481ab3afd0f392207af4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:07.106343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc951f09d39481ab3afd0f392207af4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:07.108695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:07.197489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc951f09d39481ab3afd0f392207af4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:07.238560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc951f09d39481ab3afd0f392207af4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:07.273369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc951f09d39481ab3afd0f392207af4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:07.297328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc951f09d39481ab3afd0f392207af4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:07.298315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc951f09d39481ab3afd0f392207af4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:07.323454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 887/1756] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:06 2022] DELETE /identity/v3/users/9bc951f09d39481ab3afd0f392207af4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:07.328218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:07.329088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:07.329367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:07.358686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:07.361347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:07.477001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 870/1757] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:07.486349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:07.488860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:07.503379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:07.639634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c753e-7092-4713-a1f3-68d552861f0e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:07.653440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:07.654523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:07.654965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:07.655635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:07.673324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:07.675179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54927053-1f0b-4ccc-abfc-cb7e1b88df74 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:07.782121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 888/1758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:08.187455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:08.189139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:08.215374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa804-f44c-466c-8952-1130ca011912 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:08.234008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:08.236252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:08.238043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:08.238536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] PATH_INFO: `/v3/users/865d81195b6c4611917ef321bbab6455` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:08.243029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865d81195b6c4611917ef321bbab6455)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:08.248732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:08.372158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:08.372892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865d81195b6c4611917ef321bbab6455'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:08.374836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865d81195b6c4611917ef321bbab6455'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:08.375802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865d81195b6c4611917ef321bbab6455'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:08.419651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865d81195b6c4611917ef321bbab6455'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:08.431033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-5706c060-730c-482e-bcf9-46f4bd638817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865d81195b6c4611917ef321bbab6455'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:08.482024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 871/1759] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:08 2022] DELETE /identity/v3/users/865d81195b6c4611917ef321bbab6455 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:08.482514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:08.482992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:08.483491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:08.550761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:08.553737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:08.580045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:08.580423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:08.580885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:08.766398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:08.770384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:08.795848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 889/1760] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:08.905770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 872/1761] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:03:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:08.917915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:08.921069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:09.132502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7954353-687e-433f-abd0-bcf6aa0c7865 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:09.142034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:09.143626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:09.143626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:09.143626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:09.170273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:09.172255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6675d-ba4f-490c-92ee-737719669ac4 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:09.241417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 890/1762] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:09.254918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:09.259262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:09.601560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adaaaa-87c0-48ee-931d-3e23305301fc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:09.601560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:09.602311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:09.602437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:09.603457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:09.624633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:09.626426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:09.637896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:09.647359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 873/1763] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:03:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:03:10.963558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500894e8-0f77-430d-9b41-90b174978ace None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:10.964227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500894e8-0f77-430d-9b41-90b174978ace None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:10.964439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500894e8-0f77-430d-9b41-90b174978ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:11.055423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-500894e8-0f77-430d-9b41-90b174978ace None None] MFA Rules not processed for user `3a40f55377b64f6f93784eefaf28152e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:11.057185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-500894e8-0f77-430d-9b41-90b174978ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:11.360795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:11.403045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 891/1764] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:11.521780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:11.523755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:11.587121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e4cd-c609-47ad-874f-d54b4272a56b service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:11.597313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:11.598093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:11.598371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:11.609148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] PATH_INFO: `/v3/projects/3ff94224f4e44f1ba85a37b12635e304` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:11.609779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff94224f4e44f1ba85a37b12635e304)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:11.614276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:11.769656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff94224f4e44f1ba85a37b12635e304', 'request_id': 'req-ce843081-f5ed-4153-9a75-fc2c5f8e541c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:11.788307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff94224f4e44f1ba85a37b12635e304', 'request_id': 'req-ce843081-f5ed-4153-9a75-fc2c5f8e541c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:11.809151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff94224f4e44f1ba85a37b12635e304', 'request_id': 'req-ce843081-f5ed-4153-9a75-fc2c5f8e541c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:11.831066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] The token cache is being invalidate because project 3ff94224f4e44f1ba85a37b12635e304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:11.831882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:11.832758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:11.834996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 874/1765] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:11 2022] DELETE /identity/v3/projects/3ff94224f4e44f1ba85a37b12635e304 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:11.840497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500894e8-0f77-430d-9b41-90b174978ace None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:11.857257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500894e8-0f77-430d-9b41-90b174978ace None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:11.991874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:11.992772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:11.993208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:11.994203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] PATH_INFO: `/v3/users/e8d730881eab425db648ab854ebaa639` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:11.996606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d730881eab425db648ab854ebaa639)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:12.001324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:12.123211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d730881eab425db648ab854ebaa639'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.145754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:12.151803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:12.172643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d730881eab425db648ab854ebaa639'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.205723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce843081-f5ed-4153-9a75-fc2c5f8e541c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:12.229149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:12.229917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:12.230203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:12.230480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] PATH_INFO: `/v3/projects/95d1b8374d7c477eb89d67ae84532aaf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:12.242820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d1b8374d7c477eb89d67ae84532aaf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:12.249016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d730881eab425db648ab854ebaa639'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.255260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:12.285878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d730881eab425db648ab854ebaa639'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.290573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d730881eab425db648ab854ebaa639'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.322450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 892/1766] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:11 2022] DELETE /identity/v3/users/e8d730881eab425db648ab854ebaa639 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:12.435517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d1b8374d7c477eb89d67ae84532aaf', 'request_id': 'req-b275421b-9d21-44dd-9d70-6ea08e56fdbc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.462533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d1b8374d7c477eb89d67ae84532aaf', 'request_id': 'req-b275421b-9d21-44dd-9d70-6ea08e56fdbc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.483540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d1b8374d7c477eb89d67ae84532aaf', 'request_id': 'req-b275421b-9d21-44dd-9d70-6ea08e56fdbc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.515135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] The token cache is being invalidate because project 95d1b8374d7c477eb89d67ae84532aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:12.515754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.519988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:12.523148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 875/1767] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:12 2022] DELETE /identity/v3/projects/95d1b8374d7c477eb89d67ae84532aaf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:12.527913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:12.530971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:12.882918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc252cb-4cec-43f9-a878-ad40b7f0a823 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:12.963371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:13.088615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:13.090782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:13.091206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:13.091627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] PATH_INFO: `/v3/users/3e151f29c601461ea78267225c220ea1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:13.094044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e151f29c601461ea78267225c220ea1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:13.096469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:13.234618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e151f29c601461ea78267225c220ea1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.255579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:13.257498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:13.268884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e151f29c601461ea78267225c220ea1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.280693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275421b-9d21-44dd-9d70-6ea08e56fdbc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:13.294256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:13.295198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:13.295623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:13.296044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] PATH_INFO: `/v3/users/23fb3bb0bc1441158dc2ffac03c0d22f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:13.300163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23fb3bb0bc1441158dc2ffac03c0d22f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:13.302859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:13.347264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e151f29c601461ea78267225c220ea1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.382272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e151f29c601461ea78267225c220ea1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.386907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1baff02f-3b9a-4240-88bf-7aa3416a3fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e151f29c601461ea78267225c220ea1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.424837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 893/1768] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:12 2022] DELETE /identity/v3/users/3e151f29c601461ea78267225c220ea1 => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:13.429361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23fb3bb0bc1441158dc2ffac03c0d22f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.433108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23fb3bb0bc1441158dc2ffac03c0d22f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.437462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23fb3bb0bc1441158dc2ffac03c0d22f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.461089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:13.470384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23fb3bb0bc1441158dc2ffac03c0d22f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.492220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1957946d-02e5-4774-b74c-b17a35389e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23fb3bb0bc1441158dc2ffac03c0d22f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:13.525249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 876/1769] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:13 2022] DELETE /identity/v3/users/23fb3bb0bc1441158dc2ffac03c0d22f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:13.530069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:13.530188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:13.530447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:13.580659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:13.581965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:13.599206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:13.599493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:13.599768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:13.667124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:13.668427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:13.787250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 894/1770] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:13.862425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 877/1771] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Aug 17 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:13.906417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:13.909375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:13.932911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517795d2-147c-4739-80cd-c71b483bcc58 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:13.944955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:13.945653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:13.945936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:13.946205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:13.967395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:13.971629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:13.994745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 895/1772] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:14.775001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:14.776936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:14.815589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008def03-44b1-4227-b5e9-d1ae9a5bc8ae None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:14.820152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:14.821756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:14.832880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:14.835137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:14.836262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:14.837324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] PATH_INFO: `/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:14.841199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a0b5436b00c46b98b67fd43dd3289b1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:14.847714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:14.849401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6cc35-6e9c-4a98-9b1d-ab6f4945d2c2 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:14.871025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:14.873084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:14.873425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:14.877737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] PATH_INFO: `/v3/projects/aedef472a644454e8c8d86536aa9d63c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:14.881615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aedef472a644454e8c8d86536aa9d63c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:14.889436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:15.021846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a0b5436b00c46b98b67fd43dd3289b1', 'request_id': 'req-8c568214-3b8e-4b65-b332-7fd5de085286'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.050857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a0b5436b00c46b98b67fd43dd3289b1', 'request_id': 'req-8c568214-3b8e-4b65-b332-7fd5de085286'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.064063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:15.082424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a0b5436b00c46b98b67fd43dd3289b1', 'request_id': 'req-8c568214-3b8e-4b65-b332-7fd5de085286'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.144515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] The token cache is being invalidate because project 0a0b5436b00c46b98b67fd43dd3289b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:15.148218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.152571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.155238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aedef472a644454e8c8d86536aa9d63c', 'request_id': 'req-d8b50d4e-c8ed-42f4-a859-00c37165372b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.160822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 878/1773] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:14 2022] DELETE /identity/v3/projects/0a0b5436b00c46b98b67fd43dd3289b1 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:15.163421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:15.177498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:15.192800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aedef472a644454e8c8d86536aa9d63c', 'request_id': 'req-d8b50d4e-c8ed-42f4-a859-00c37165372b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.232623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aedef472a644454e8c8d86536aa9d63c', 'request_id': 'req-d8b50d4e-c8ed-42f4-a859-00c37165372b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.280424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] The token cache is being invalidate because project aedef472a644454e8c8d86536aa9d63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:15.287531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.292121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.297391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 896/1774] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:14 2022] DELETE /identity/v3/projects/aedef472a644454e8c8d86536aa9d63c => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:15.298657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:15.300657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:15.458736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c568214-3b8e-4b65-b332-7fd5de085286 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:15.493785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:15.495671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b50d4e-c8ed-42f4-a859-00c37165372b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:15.507777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:15.510825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:15.512429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:15.512909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:15.523480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:15.564066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:15.578745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:15.608373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 897/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:15.616108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:15.639679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ca9a2-23f1-45ef-a2cf-9927da613af4 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:15.649888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:15.650836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:15.650970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:15.652891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:15.653273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:15.653631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:15.653910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] PATH_INFO: `/v3/projects/082124a3989c4e928bbda32cceb7d286` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:15.654484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:15.657279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082124a3989c4e928bbda32cceb7d286)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:15.664900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:15.683120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:15.685563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:15.723232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 898/1776] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:15.725602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:15.729405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:15.757785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615f3b9-13af-40cc-bd6b-777af328b935 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:15.773066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:15.774912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:15.775936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:15.776792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:15.794845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:15.801465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ddad2-ea16-45f3-ac49-961a742f997a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:15.824494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 899/1777] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:15.827108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:15.827398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:15.827669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:15.889855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:15.891203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:15.892736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082124a3989c4e928bbda32cceb7d286', 'request_id': 'req-41266ec1-6021-4cc0-a062-94fa1fd53979'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.929313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082124a3989c4e928bbda32cceb7d286', 'request_id': 'req-41266ec1-6021-4cc0-a062-94fa1fd53979'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:15.967462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082124a3989c4e928bbda32cceb7d286', 'request_id': 'req-41266ec1-6021-4cc0-a062-94fa1fd53979'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.018039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] The token cache is being invalidate because project 082124a3989c4e928bbda32cceb7d286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:16.025713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.031906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-41266ec1-6021-4cc0-a062-94fa1fd53979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.040626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 879/1778] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:15 2022] DELETE /identity/v3/projects/082124a3989c4e928bbda32cceb7d286 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:16.042942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881da49-4e5c-43e1-815d-df6e2287d6a8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.049345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881da49-4e5c-43e1-815d-df6e2287d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.056933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881da49-4e5c-43e1-815d-df6e2287d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.104279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2881da49-4e5c-43e1-815d-df6e2287d6a8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:16.112038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 900/1779] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:16.113483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.116235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.124022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2881da49-4e5c-43e1-815d-df6e2287d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.143179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b2fd6-7717-421d-9a7d-a65b28b80710 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.148303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 880/1780] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:16.149944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d037-333e-4863-9e21-70cfa37660ed None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.150337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d037-333e-4863-9e21-70cfa37660ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.150821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d037-333e-4863-9e21-70cfa37660ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.155939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.156698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.156974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.157678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] PATH_INFO: `/v3/users/4a962757b4d54c7685fff119ecdedbcd` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.160005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a962757b4d54c7685fff119ecdedbcd)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:16.166176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:16.177431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e107d037-333e-4863-9e21-70cfa37660ed None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:16.203641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e107d037-333e-4863-9e21-70cfa37660ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.220198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 881/1781] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:16.231021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.231713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.232727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.269375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:16.275448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:16.287741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.325156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 882/1782] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:16.326247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.354699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a962757b4d54c7685fff119ecdedbcd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.357289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1d678-d84b-41f8-92fe-442fd5524903 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.370115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.370865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.371164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.371435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.372484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:16.373158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:16.374910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:16.381158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a962757b4d54c7685fff119ecdedbcd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.398760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 883/1783] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:16.398760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.419439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af789945-a4e8-463d-b50c-792695e48846 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.429415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.430114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.430408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.432831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.433109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a962757b4d54c7685fff119ecdedbcd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.434237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:16.435397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:16.440048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:16.444902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a962757b4d54c7685fff119ecdedbcd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.445921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a962757b4d54c7685fff119ecdedbcd'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:16.464388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 901/1784] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:16 2022] DELETE /identity/v3/users/4a962757b4d54c7685fff119ecdedbcd => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:16.465262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.471042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.476137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 884/1785] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:16.479957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.486754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:16.519942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86aead-9ab9-42a1-b236-f5da4e76290c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.677445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4626d3-5373-4272-8f13-4972d4333b64 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.683945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.686292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.687761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.688046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.689909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:16.696102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:16.696527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.697176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.697449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:16.697746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:16.698560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:16.700618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:16.708048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:16.759254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 885/1786] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/projects => generated 373 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:16.768775 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.774947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 902/1787] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/projects => generated 379 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:16.779249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:16.905441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77e73e-3b01-4d49-8d8f-a157f12b8c01 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.987347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed269d7-ba0b-4a23-8745-c18b0624763a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:16.998560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:16.999374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:16.999754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.000125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.000883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.007145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.063792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.064755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.066305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 903/1788] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/users => generated 290 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:17.069284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.069802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.070647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.071309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.073053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.090770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3c2a8-46dd-4ba4-977f-81eadcb304d8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.096949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.097742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.098123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.098528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.099286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.101705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.160743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 886/1789] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:16 2022] POST /identity/v3/users => generated 287 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:17.160972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 904/1790] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:17.162567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.164652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.205518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22f906-00e8-4cd9-b0c4-30933fd21b30 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.207637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4583c-f509-4ead-b82c-9b5a4a342803 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.224276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.225092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.225875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.226598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.226990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.227446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.227900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.228266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.229671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.230528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.243761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.246454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.248268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:17.284604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 905/1791] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:17.285755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.300405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 887/1792] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:17.311307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.327830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fb165-6c0f-473d-b021-f426c4a0f949 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.360402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0b5a3-d216-4cf0-8790-2ad3ca7f8971 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.373428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.375093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.375239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.375461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.376230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.380936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.392587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.393619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.393781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.394039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] PATH_INFO: `/v3/projects/1e31ad3861364c27876768d037293687/users/94da3e3550b44339b350f22707874d50/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.438557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e31ad3861364c27876768d037293687, user_id=94da3e3550b44339b350f22707874d50, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.443543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f66e7b-98a4-4678-82ab-0d1198ba741d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.451603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 906/1793] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:17.457630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.487455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 888/1794] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:17 2022] PUT /identity/v3/projects/1e31ad3861364c27876768d037293687/users/94da3e3550b44339b350f22707874d50/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:17.503942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622174e8-9aa1-4b6d-91b5-a378bd7e2e65 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:17.504105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.504229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.504505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.582590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] MFA Rules not processed for user `94da3e3550b44339b350f22707874d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:17.587470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:17.676621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:17.777488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 889/1795] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:17.779075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:17.804048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:17.804742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.805023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.805294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] PATH_INFO: `/v3/projects/857a411a261c45c2bc1c042ca03cfe96/users/cd657758dff74bfd9457f09f5b0815ff/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.865047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] RBAC: Authorizing `identity:create_grant(project_id=857a411a261c45c2bc1c042ca03cfe96, user_id=cd657758dff74bfd9457f09f5b0815ff, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:17.870440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7865b-a228-4642-825f-79584d61420f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:17.902335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 907/1796] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:17 2022] PUT /identity/v3/projects/857a411a261c45c2bc1c042ca03cfe96/users/cd657758dff74bfd9457f09f5b0815ff/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:17.915326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:17.915480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:17.915807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:17.975527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] MFA Rules not processed for user `cd657758dff74bfd9457f09f5b0815ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:17.976743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:17.985011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659a8b7-5bb4-4373-a14e-233d711d29ed None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.159759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 908/1797] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:18.160246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.160626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.160910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.161183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.161886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.166054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:18.176090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.190103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.281910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 890/1798] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:17 2022] POST /identity/v3/projects => generated 379 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:18.282921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.329773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad19f4b-27a0-4f91-8c81-7e06c8de11bc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.372440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:18.405504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc92c4c-f332-4352-b38b-b39a267eb965 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.424277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.425159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.425784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.426235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.427014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.429102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.467012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:18.477059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.491059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.491297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.491794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.498161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 891/1799] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:18 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:18.502392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.535305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6c6b-21be-4d7f-93c2-de0f51294689 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.559841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.560486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.560764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.561036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.561728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.563356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.587376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 892/1800] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:18.592865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.615079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c85eab8-a62d-4640-a19c-4618cc73c10b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.625009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.630278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.636158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.636959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.637240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.637515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] PATH_INFO: `/v3/projects/691af0f676454b32a04c94762e763c97/users/0f9c964a4c4b4ca99909e7f861f10d6d/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.659340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691af0f676454b32a04c94762e763c97, user_id=0f9c964a4c4b4ca99909e7f861f10d6d, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.662080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.667283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 909/1801] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:18.694375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.701602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 893/1802] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:18 2022] PUT /identity/v3/projects/691af0f676454b32a04c94762e763c97/users/0f9c964a4c4b4ca99909e7f861f10d6d/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:18.706817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:18.729490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc6e61-5e4f-4950-bcac-52648105aa4a service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.943665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.944949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2948b8c-97fb-43f1-94fe-cd2c16af4239 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:18.945700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.946215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.946788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] PATH_INFO: `/v3/users/cd657758dff74bfd9457f09f5b0815ff` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.948524 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd657758dff74bfd9457f09f5b0815ff)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.951254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:18.974664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:18.975304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:18.975578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:18.975842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:18.976958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:18.978816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:19.006781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 894/1803] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:19.007872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:19.010644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.040188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3999c41-e472-41db-902a-cb25bc9f6175 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:19.050786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:19.051341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.051703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.051851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] PATH_INFO: `/v3/projects/0bd1897121bd4c02a01d4f8487d7623e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.057307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd1897121bd4c02a01d4f8487d7623e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:19.059368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:19.112467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd657758dff74bfd9457f09f5b0815ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.153036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd657758dff74bfd9457f09f5b0815ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.168565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd1897121bd4c02a01d4f8487d7623e', 'request_id': 'req-4830e71e-7b84-4d16-a900-b150081b0c0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.188329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd1897121bd4c02a01d4f8487d7623e', 'request_id': 'req-4830e71e-7b84-4d16-a900-b150081b0c0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.201851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd1897121bd4c02a01d4f8487d7623e', 'request_id': 'req-4830e71e-7b84-4d16-a900-b150081b0c0c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.203356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:19.213382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd657758dff74bfd9457f09f5b0815ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.230046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd657758dff74bfd9457f09f5b0815ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.231640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-612541c4-0098-4a68-b416-0802913189fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd657758dff74bfd9457f09f5b0815ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.231748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] The token cache is being invalidate because project 0bd1897121bd4c02a01d4f8487d7623e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:19.232019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.239353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:19.243443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 895/1804] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:19 2022] DELETE /identity/v3/projects/0bd1897121bd4c02a01d4f8487d7623e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:19.244272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:19.247281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 910/1805] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:18 2022] DELETE /identity/v3/users/cd657758dff74bfd9457f09f5b0815ff => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:19.248724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d8800-1c20-4c4b-b072-ea7782a12c0b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.249249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d8800-1c20-4c4b-b072-ea7782a12c0b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.249728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d8800-1c20-4c4b-b072-ea7782a12c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.250895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.314987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c85d8800-1c20-4c4b-b072-ea7782a12c0b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:19.319606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c85d8800-1c20-4c4b-b072-ea7782a12c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.402426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:19.466625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830e71e-7b84-4d16-a900-b150081b0c0c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:19.485336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 911/1806] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:19.486837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4398b4-5254-4929-a321-fb527780d1b7 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.487198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4398b4-5254-4929-a321-fb527780d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.487395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4398b4-5254-4929-a321-fb527780d1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.550764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-cc4398b4-5254-4929-a321-fb527780d1b7 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:19.552076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc4398b4-5254-4929-a321-fb527780d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.576028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 912/1807] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:19.577419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85758f88-f33e-48c1-9b86-d87649b1cede None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.577730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85758f88-f33e-48c1-9b86-d87649b1cede None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.578002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85758f88-f33e-48c1-9b86-d87649b1cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.615227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-85758f88-f33e-48c1-9b86-d87649b1cede None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:19.616867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-85758f88-f33e-48c1-9b86-d87649b1cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.627430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 913/1808] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:19.628833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.629123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.629490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.631177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 914/1809] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:19.632485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:19.634924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:19.689931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:19.690758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.692426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.692717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] PATH_INFO: `/v3/projects/691af0f676454b32a04c94762e763c97/users/0f9c964a4c4b4ca99909e7f861f10d6d/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.700276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691af0f676454b32a04c94762e763c97, user_id=0f9c964a4c4b4ca99909e7f861f10d6d, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:19.707730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:19.750049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 896/1810] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:19 2022] PUT /identity/v3/projects/691af0f676454b32a04c94762e763c97/users/0f9c964a4c4b4ca99909e7f861f10d6d/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:19.763256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:19.829800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432703e9-18e8-47f5-bbbc-42e8c294f1d8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:19.899186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba983461-2567-45d4-8b51-33411a1b31b4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:19.910778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:19.912201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.912744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.913187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.914205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:19.914660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:19.917795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873614d3-baa0-4ef4-be47-87ed47d0e3b8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:19.960963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:19.965461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.965895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 897/1811] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:19.965986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.965986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.966133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:19.966133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:19.966133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:19.994212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:19.996172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65afac4-282c-4f63-9436-d5e637f65457 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.014561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:20.016075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:20.016970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 915/1812] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:20.018573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.019154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.019722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.083286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] MFA Rules not processed for user `0f9c964a4c4b4ca99909e7f861f10d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:20.085585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:20.135208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:20.220300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 898/1813] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:03:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:20.221278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.248687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed28b8-973c-4ecf-9d2b-4fa5df08c915 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.254311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.254953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.255224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.255480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.256225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.259248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.274608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 916/1814] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:20.290832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.292718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:20.293966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:20.330986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 899/1815] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:20 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:20.335288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.411691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb739e-acf6-4c33-91f6-8f282a48ea28 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.555719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a9a0c-b37f-4874-95ce-6688f67fabc9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.570267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.571051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.571342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.571623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.572273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.580034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df35b8-77e6-44d3-9c05-8c3fdb2be72a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.586393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.586889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.587168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.587438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.635652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 900/1816] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:20 2022] POST /identity/v3/users => generated 278 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:20.640763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090aed-5634-455a-9052-213afd63774c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.641058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090aed-5634-455a-9052-213afd63774c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.641692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090aed-5634-455a-9052-213afd63774c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.647421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 901/1817] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:03:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:20.648306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91090aed-5634-455a-9052-213afd63774c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.681771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91090aed-5634-455a-9052-213afd63774c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.691768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.692481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.700231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.700846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.702982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.704829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.735036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 902/1818] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:20.740159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.760824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b38911-8695-4267-846b-a66a1531b023 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.762002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.763888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.776022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 917/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:20.779125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.779769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.780056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.780464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.781130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.783454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f95466-b408-4c2f-adf3-431ff0edcb21 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.814610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:20.815496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 903/1820] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:20.828982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300f488-7843-4768-823a-6865acdba0b6 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:20.841675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:20.844045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.851501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.851501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] PATH_INFO: `/v3/projects/0ace49ae706e4cc4bfd0805b99f236f0/users/02f6186669f74e85a51e53c58f45d9af/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.900754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ace49ae706e4cc4bfd0805b99f236f0, user_id=02f6186669f74e85a51e53c58f45d9af, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:20.904919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e93d52-3142-4b26-894c-7df3a4b39249 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:20.909611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.909832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.910343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.933851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:20.940332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 918/1821] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:20 2022] PUT /identity/v3/projects/0ace49ae706e4cc4bfd0805b99f236f0/users/02f6186669f74e85a51e53c58f45d9af/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:20.949094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:20.950478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:20.950931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:20.980692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:20.981975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:21.023594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] MFA Rules not processed for user `02f6186669f74e85a51e53c58f45d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:21.025177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:21.282875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 904/1822] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:21.284609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.291133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:21.311171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 919/1823] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:21.316251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.316743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e612b0c-3392-4767-8b5d-9c16fb36e5c9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.330774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.334436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.334752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.335036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.372225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.374091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.450409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 905/1824] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:21.469822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.483227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:21.563775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bc2ba-f999-405f-ada7-fc0e7663ba9e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.575712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41070a06-d767-41da-b4b2-5b8d7e38b630 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.585421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.588628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.588944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.589222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.596865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.597122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.597122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.597122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.597122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.600856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.622712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.622712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.652061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 906/1825] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:21.678652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.679232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 920/1826] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:21 2022] POST /identity/v3/projects => generated 355 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:21.696338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109766cd-0a72-4eb6-a6e3-b442efa48d9c service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.731527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:21.765919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.824504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.825977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.827047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.839131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.842483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.848191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.903978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 907/1827] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:21 2022] POST /identity/v3/users => generated 284 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:21.908014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.933378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7df36-3a26-4ddf-ac52-c306918cf0c4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.939562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0261c9-ce9e-4fd2-ada6-a0b1a6681b15 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:21.944165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.947018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.947531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.948966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.949799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.951160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:21.951830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:21.952109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:21.952382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] PATH_INFO: `/v3/users/94da3e3550b44339b350f22707874d50` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:21.955099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.955279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=94da3e3550b44339b350f22707874d50)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:21.957240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:21.983128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 908/1828] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:21.985710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:21.996775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:22.003954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b0495-37a5-4899-adfb-c87e77dbcdad admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:22.013250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:22.015448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.016159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.017219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] PATH_INFO: `/v3/projects/6a9b9b66759840d18057c3c95c802177/users/0e695d0d73a7489f89d5aafaf54bc5bf/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.074511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9b9b66759840d18057c3c95c802177, user_id=0e695d0d73a7489f89d5aafaf54bc5bf, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:22.084675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:22.090756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94da3e3550b44339b350f22707874d50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.101363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94da3e3550b44339b350f22707874d50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.127019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94da3e3550b44339b350f22707874d50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.142691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 909/1829] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:21 2022] PUT /identity/v3/projects/6a9b9b66759840d18057c3c95c802177/users/0e695d0d73a7489f89d5aafaf54bc5bf/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:22.146591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94da3e3550b44339b350f22707874d50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.146710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:22.147306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6faeed85-c8cc-4755-9af3-8cf2722938fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94da3e3550b44339b350f22707874d50'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.159929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 921/1830] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:21 2022] DELETE /identity/v3/users/94da3e3550b44339b350f22707874d50 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:22.164186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.164695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.164976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.202688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:22.204918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:22.290669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab534f-c33a-4445-9e55-cde9d06d370f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:22.367646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 922/1831] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:22.414721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:22.415893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.415893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.416075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.419704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:22.421255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:22.427580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:22.443390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:22.463482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 910/1832] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:22.467662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:22.470051 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:22.492140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da167ac-8596-40f0-89ca-306972a6e3db admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:22.501921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:22.506015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.507807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.508278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] PATH_INFO: `/v3/projects/6a9b9b66759840d18057c3c95c802177/users/0e695d0d73a7489f89d5aafaf54bc5bf/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.514290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9b9b66759840d18057c3c95c802177, user_id=0e695d0d73a7489f89d5aafaf54bc5bf, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:22.518950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:22.543528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 911/1833] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:22 2022] PUT /identity/v3/projects/6a9b9b66759840d18057c3c95c802177/users/0e695d0d73a7489f89d5aafaf54bc5bf/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:22.544596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:22.546463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:22.563910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cef5f-526a-4398-82ee-961e90bb074b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:22.598518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:22.698161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:22.698979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.699233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.699507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.718357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:22.781306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e084393-8ffa-406a-ba31-013aa608483b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:22.787703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:22.788356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.788776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.789052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] PATH_INFO: `/v3/projects/857a411a261c45c2bc1c042ca03cfe96` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.790237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857a411a261c45c2bc1c042ca03cfe96)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:22.792299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:22.862569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857a411a261c45c2bc1c042ca03cfe96', 'request_id': 'req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.869221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857a411a261c45c2bc1c042ca03cfe96', 'request_id': 'req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.881611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857a411a261c45c2bc1c042ca03cfe96', 'request_id': 'req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.891239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:22.894282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fc881-f398-45d8-9534-c5339fa485b6 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:22.894580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] The token cache is being invalidate because project 857a411a261c45c2bc1c042ca03cfe96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:22.894950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.895749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7bc048a6-8d9c-4cd4-900d-c84ccd1f6173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:22.901494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 912/1834] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:22 2022] DELETE /identity/v3/projects/857a411a261c45c2bc1c042ca03cfe96 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:22.902739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.903027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.903304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.911722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 923/1835] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:22.914541 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da3d8-28f3-4f79-b5bc-6c7a94cea286 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:22.914838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da3d8-28f3-4f79-b5bc-6c7a94cea286 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:22.915155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647da3d8-28f3-4f79-b5bc-6c7a94cea286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:22.937994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-647da3d8-28f3-4f79-b5bc-6c7a94cea286 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:22.940598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-647da3d8-28f3-4f79-b5bc-6c7a94cea286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:22.949425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] MFA Rules not processed for user `0e695d0d73a7489f89d5aafaf54bc5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:22.950876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.055504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 913/1836] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:23.083123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 924/1837] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:23.083427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.083427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.085374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.085821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.086239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.109454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:23.111109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.121233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:23.123101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 925/1838] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:23.127529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.139275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5840e90-d8a7-461f-be66-0be2d4135105 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.152918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.153622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.153906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.154174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.154863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:23.155162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.157513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.176509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 926/1839] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:03:23.177286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.188475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076d67b-11f4-45c9-a36a-ce6eaaa61ef2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.195402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.196056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.196377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.196794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.197432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.198346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297b781-8c9e-4885-b7b6-eb10e0dbce6c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.200031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.206584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.207312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.207751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.208405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.220628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.222344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.234178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 914/1840] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:23.238190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 927/1841] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:23 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:23.242615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.254168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e1bde-da54-4bb0-9a97-7a02c4e747fa service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.307406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.309953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.353397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.354265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.354412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.354682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.355318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.359695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.388144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 915/1842] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:23 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:23.388144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.388144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.401919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1419f-3fae-4e09-8107-ffcd80e7564a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.406701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.407340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.407738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.408012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] PATH_INFO: `/v3/projects/1e31ad3861364c27876768d037293687` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.409136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e31ad3861364c27876768d037293687)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.411107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.412633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:23.448804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8e1bb-fdf6-46fa-9e65-2894b8e4bb4e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.460564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e31ad3861364c27876768d037293687', 'request_id': 'req-dad4f034-421d-4b2a-97dc-0094cf815307'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.466161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e31ad3861364c27876768d037293687', 'request_id': 'req-dad4f034-421d-4b2a-97dc-0094cf815307'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.472998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e31ad3861364c27876768d037293687', 'request_id': 'req-dad4f034-421d-4b2a-97dc-0094cf815307'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.487261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] The token cache is being invalidate because project 1e31ad3861364c27876768d037293687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:23.487546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.488187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.490089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 928/1843] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:23 2022] DELETE /identity/v3/projects/1e31ad3861364c27876768d037293687 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:23.491033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.492311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.504401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.505026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.505357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.505660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.507482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.533690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:23.563285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4f034-421d-4b2a-97dc-0094cf815307 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.572695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.573571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.574170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.574294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.574468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.574718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.575400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.577098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.583069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 916/1844] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:23.583874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.585242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.590366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 929/1845] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:23.591416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.601209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a278a4-14ea-4d00-9c23-03145b668562 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.601335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb94ce-356e-4e37-9531-25571928859a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.605923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.606078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.606551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.606663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.606824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.606929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.607094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] PATH_INFO: `/v3/users/0f9c964a4c4b4ca99909e7f861f10d6d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.607190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.607765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.608225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9c964a4c4b4ca99909e7f861f10d6d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.609394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.610141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.621574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 930/1846] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:23.624203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.636268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c69fd-83cd-4040-806e-9ede255798cf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.642235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.643134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.643498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.644042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] PATH_INFO: `/v3/projects/12a4bd2c837d4d00a0dadec048e04fde/users/dd1bff35591447db814f07bbbbad7e3d/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.664802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a4bd2c837d4d00a0dadec048e04fde, user_id=dd1bff35591447db814f07bbbbad7e3d, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:23.668436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93db9d7-23a2-42b0-9319-eb3587247945 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:23.673177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9c964a4c4b4ca99909e7f861f10d6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.673930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9c964a4c4b4ca99909e7f861f10d6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.674709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9c964a4c4b4ca99909e7f861f10d6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.694223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 931/1847] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:23 2022] PUT /identity/v3/projects/12a4bd2c837d4d00a0dadec048e04fde/users/dd1bff35591447db814f07bbbbad7e3d/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:23.699618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.699947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.700336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.714340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9c964a4c4b4ca99909e7f861f10d6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.728670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9c964a4c4b4ca99909e7f861f10d6d'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:23.754806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 917/1848] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:23 2022] DELETE /identity/v3/users/0f9c964a4c4b4ca99909e7f861f10d6d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:23.773834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] MFA Rules not processed for user `dd1bff35591447db814f07bbbbad7e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:23.775299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.828263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:23.865733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 932/1849] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:23.880155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:23.881648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:23.989438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502a4a5-cabc-4891-9ebd-a24f4379a93a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:23.989438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:23.989438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:23.989438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:23.989793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:23.998962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.000242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa4bc8-7dea-47f8-bf76-34eb309cf45d service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.011108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 918/1850] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:24.242868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.244862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:24.349044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c13f9e7-aec7-45f5-8f43-3e153520926b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.353526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.354163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.354583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.354951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] PATH_INFO: `/v3/projects/691af0f676454b32a04c94762e763c97` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.356137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=691af0f676454b32a04c94762e763c97)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.358183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.408913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691af0f676454b32a04c94762e763c97', 'request_id': 'req-e2c8e35d-172d-4934-91db-69d41ae3a1d2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:24.417509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691af0f676454b32a04c94762e763c97', 'request_id': 'req-e2c8e35d-172d-4934-91db-69d41ae3a1d2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:24.424830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691af0f676454b32a04c94762e763c97', 'request_id': 'req-e2c8e35d-172d-4934-91db-69d41ae3a1d2'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:24.442091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] The token cache is being invalidate because project 691af0f676454b32a04c94762e763c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:24.442399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:24.443023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e2c8e35d-172d-4934-91db-69d41ae3a1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:24.444611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 933/1851] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:24 2022] DELETE /identity/v3/projects/691af0f676454b32a04c94762e763c97 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:24.450673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.451179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.451269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.472370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:24.473818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:24.496601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:24.553011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 919/1852] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:24.560979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.561337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.561669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.587393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:24.588675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:24.596597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 934/1853] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:24.601141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.610951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188cec4-fbc7-48be-886e-0bb9c8d4721d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.616492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.617122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.617402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.617731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.618399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:24.618693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.620404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.633113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 920/1854] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:24.637839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.647107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bc99a-71b6-4aef-9078-35133dffedf1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.651927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.652497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.652805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.653072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.653729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.655691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.685458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 935/1855] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:24 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:24.689841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.700143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41b97-fb96-43ec-a63d-9f4b682dce7a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.769706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.770338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.770680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.770873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.771511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.773444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.799966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 921/1856] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:24 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:24.804411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.816196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba536a4-e2e3-4e21-bcdb-d62dd56d792c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.821160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.821884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.822222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.822551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.823378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.825113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.835813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 936/1857] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:24.840579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.850745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28638-5fc6-4226-9578-759678cd9c9d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.856238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.856851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.857128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.857394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.858063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.859690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3bfce-c141-48b1-bd8c-19cfa62a748c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.870681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 922/1858] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:24.875295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:24.886423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf979241-c0dd-428b-a94b-1e97afb629a1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:24.892535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:24.893192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.893462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.893729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] PATH_INFO: `/v3/projects/2080ad21b1814290b878dc25c4c6bcc6/users/2c0248c0023a414f97254264f51c19e2/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:24.911260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2080ad21b1814290b878dc25c4c6bcc6, user_id=2c0248c0023a414f97254264f51c19e2, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:24.914125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:24.960185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 937/1859] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:24 2022] PUT /identity/v3/projects/2080ad21b1814290b878dc25c4c6bcc6/users/2c0248c0023a414f97254264f51c19e2/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:24.974236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:24.974602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:24.974799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.012265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] MFA Rules not processed for user `2c0248c0023a414f97254264f51c19e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:25.013805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:25.085489 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 923/1860] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:25.101324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:25.104223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:25.127852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:25.170607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8733d5f7-a295-4a3d-a952-1f1d3d6e253f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:25.174684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:25.175372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.175786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.176057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.184497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:25.186032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d81179-e268-4cf2-9fd3-c84c403a102d service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:25.194981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 938/1861] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:25.214313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:25.231299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:25.277167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4b155-c238-44a9-92f1-b98d7685ea72 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:25.283140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:25.284640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.284918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.285186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] PATH_INFO: `/v3/users/2c0248c0023a414f97254264f51c19e2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.286461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0248c0023a414f97254264f51c19e2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:25.288470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:25.349914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0248c0023a414f97254264f51c19e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.351202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0248c0023a414f97254264f51c19e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.351832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0248c0023a414f97254264f51c19e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.370932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0248c0023a414f97254264f51c19e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.379822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0248c0023a414f97254264f51c19e2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.393914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 924/1862] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:25 2022] DELETE /identity/v3/users/2c0248c0023a414f97254264f51c19e2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:25.398676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.399037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.399850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.418784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:25.419988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:25.484880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 939/1863] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:25.496573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:25.498691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:25.584671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f7929-e52d-4bd5-917d-13f2350dc90f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:25.588263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:25.588888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.589157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.589428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.599161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:25.600587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fea6c-4c53-41ea-8f83-227941015e47 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:25.610323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 925/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:25.892118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:25.893656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:25.907078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73badd-6057-44cf-8f66-f02e52f1f67c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:25.912224 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:25.912849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.913574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.913574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] PATH_INFO: `/v3/projects/2080ad21b1814290b878dc25c4c6bcc6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:25.914552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2080ad21b1814290b878dc25c4c6bcc6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:25.916475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:25.964246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2080ad21b1814290b878dc25c4c6bcc6', 'request_id': 'req-c24033b7-9f22-4807-b951-a6fce31c6326'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.969812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2080ad21b1814290b878dc25c4c6bcc6', 'request_id': 'req-c24033b7-9f22-4807-b951-a6fce31c6326'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.976335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2080ad21b1814290b878dc25c4c6bcc6', 'request_id': 'req-c24033b7-9f22-4807-b951-a6fce31c6326'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.990882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] The token cache is being invalidate because project 2080ad21b1814290b878dc25c4c6bcc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:25.991159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.991791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c24033b7-9f22-4807-b951-a6fce31c6326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:25.993337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 940/1865] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:25 2022] DELETE /identity/v3/projects/2080ad21b1814290b878dc25c4c6bcc6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:25.998452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:25.998741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:25.999015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.018351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:26.019555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:26.052049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:26.079565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 926/1866] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:26.087609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.087886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.088153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.108492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:26.109726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:26.117918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 941/1867] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:26.122611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.132372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458125-678d-4fa4-8537-e0b287b6afa3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.137354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.138011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.138265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.138532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.139172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:26.139475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.141053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.153041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 927/1868] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:03:26.157793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.167105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847735e1-c15c-4c01-812b-92fe38a5f99f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.172613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.173310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.173612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.173881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.174519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.176420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.209219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 942/1869] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:26.213673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.225714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e321e-9a34-42d9-a2f0-f2711d65c38a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.285260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.285912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.286187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.286458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.287099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.289152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.312631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 928/1870] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:26.316971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.324452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:26.329362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b97ad-1562-4293-8fa7-7734003fe7d8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.336689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.337327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.338393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.338703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.339736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.341769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.356947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 943/1871] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:26.358247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.370114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef85f5-5ff5-41ce-8195-341c794b0865 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.375370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.375995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.376276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.376553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.377248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.378908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e6873-a9b9-4ff7-8326-e82580846381 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.394279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 929/1872] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:26.398476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.408349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda99a1-99b0-489a-9020-e61c53c98e62 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.412886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.413526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.413822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.414085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] PATH_INFO: `/v3/projects/1686c2699355403d8148c24731dd5a75/users/f95f6586193945d4a231cab0026782c4/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.428749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1686c2699355403d8148c24731dd5a75, user_id=f95f6586193945d4a231cab0026782c4, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.431444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.445975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 944/1873] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:26 2022] PUT /identity/v3/projects/1686c2699355403d8148c24731dd5a75/users/f95f6586193945d4a231cab0026782c4/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:26.457926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.458180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.458475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.489673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] MFA Rules not processed for user `f95f6586193945d4a231cab0026782c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:26.490841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:26.557796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 930/1874] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:26.562602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.642862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07f1f4-b708-4791-b8c1-db728c2d0716 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.648294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.648910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.649187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.649615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.650381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.652363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.682528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 945/1875] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:26.686767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.698531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa665434-986d-4786-bfdb-e38e7dd46e88 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.731654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:26.763358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.763894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.764171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.764442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.765086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.767094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.793578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 931/1876] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:26 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:26.794915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.806245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc9046-50f5-4f02-b5c0-c04ec86c0d05 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.811883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.812501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.812777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.813043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.813730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.815362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.827877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 946/1877] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:26.833196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.843647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0593-8200-4b43-9a76-dce45dc77cb5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.848552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.849306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.849610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.849903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] PATH_INFO: `/v3/projects/5e6c4cb5fc004864bab12042e42c62c6/users/e4afebe92e80414ebe9bbfd403f6356f/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.867015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6c4cb5fc004864bab12042e42c62c6, user_id=e4afebe92e80414ebe9bbfd403f6356f, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.869714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.886544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 932/1878] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:26 2022] PUT /identity/v3/projects/5e6c4cb5fc004864bab12042e42c62c6/users/e4afebe92e80414ebe9bbfd403f6356f/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:26.890665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.900619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cd3d-7e29-498f-96d6-f82fefc8b900 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:26.970544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:26.971197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:26.971474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:26.971744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:26.972430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:26.974178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1d5be-3ec2-4806-bf6f-433d930d3df1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:26.984357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 947/1879] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:26.988689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:26.999714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80069e55-2693-4041-b743-7ff16d08fc01 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:27.004295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:27.004954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:27.005235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:27.005507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] PATH_INFO: `/v3/projects/5e6c4cb5fc004864bab12042e42c62c6/users/e4afebe92e80414ebe9bbfd403f6356f/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:27.007483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6c4cb5fc004864bab12042e42c62c6, user_id=e4afebe92e80414ebe9bbfd403f6356f, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:27.010160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:27.020975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 933/1880] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:26 2022] PUT /identity/v3/projects/5e6c4cb5fc004864bab12042e42c62c6/users/e4afebe92e80414ebe9bbfd403f6356f/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:27.031562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:27.031848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:27.032137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:27.061246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] MFA Rules not processed for user `e4afebe92e80414ebe9bbfd403f6356f`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:27.062501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:27.091521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:27.137916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 948/1881] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:27.155497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:27.158471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:27.223124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf5c40-472d-4160-ab2b-e4bac14f62e2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:27.227190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:27.227794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:27.228070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:27.228345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:27.289623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:27.290971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba5e46-a739-477c-b15b-6842768103a5 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:27.300524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 934/1882] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:35.356420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:35.472241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9670-e852-4729-930e-cfbf45d36427 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:35.476417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:35.476807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:35.477345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:35.477345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] PATH_INFO: `/v3/users/f95f6586193945d4a231cab0026782c4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:35.482259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95f6586193945d4a231cab0026782c4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:35.482259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:35.559335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95f6586193945d4a231cab0026782c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:35.569199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95f6586193945d4a231cab0026782c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:35.584759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95f6586193945d4a231cab0026782c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:35.594706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95f6586193945d4a231cab0026782c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:35.595442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95f6586193945d4a231cab0026782c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:35.620555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 949/1883] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:35 2022] DELETE /identity/v3/users/f95f6586193945d4a231cab0026782c4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:35.624096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:35.624614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:35.624693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:35.670023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:35.671277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:35.724924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:35.777373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 935/1884] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:35.788216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:35.790238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:35.851736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:35.873790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984abdc-533b-49c8-95cf-674f4cc78647 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:35.877680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:35.878333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:35.878566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:35.878828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:35.888585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:35.890305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:35.900401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 950/1885] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:36.185413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:36.187544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:36.206020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927caab6-073e-4c84-ba07-e50ab522de51 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:36.213225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:36.214080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:36.215038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:36.215767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] PATH_INFO: `/v3/projects/1686c2699355403d8148c24731dd5a75` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:36.217910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1686c2699355403d8148c24731dd5a75)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:36.221975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:36.275054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1686c2699355403d8148c24731dd5a75', 'request_id': 'req-8de4e636-60d0-4e5e-9527-e8f72a2174d7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.280744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1686c2699355403d8148c24731dd5a75', 'request_id': 'req-8de4e636-60d0-4e5e-9527-e8f72a2174d7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.290843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1686c2699355403d8148c24731dd5a75', 'request_id': 'req-8de4e636-60d0-4e5e-9527-e8f72a2174d7'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.303371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] The token cache is being invalidate because project 1686c2699355403d8148c24731dd5a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:36.303653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.304335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.306075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 936/1886] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:36 2022] DELETE /identity/v3/projects/1686c2699355403d8148c24731dd5a75 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:36.310435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:36.319846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e1e20-39c2-44a6-8f51-985af7401b0a service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:36.386423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:36.387110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:36.387260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:36.387488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] PATH_INFO: `/v3/users/e4afebe92e80414ebe9bbfd403f6356f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:36.388696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4afebe92e80414ebe9bbfd403f6356f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:36.390772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:36.444945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4afebe92e80414ebe9bbfd403f6356f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.452304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4afebe92e80414ebe9bbfd403f6356f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.466215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4afebe92e80414ebe9bbfd403f6356f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.479689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4afebe92e80414ebe9bbfd403f6356f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.479689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ba25279-01a5-4d68-8f50-7ee7c3a6429e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4afebe92e80414ebe9bbfd403f6356f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:36.489646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 951/1887] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:36 2022] DELETE /identity/v3/users/e4afebe92e80414ebe9bbfd403f6356f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:36.959469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:36.960859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:36.987639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4e636-60d0-4e5e-9527-e8f72a2174d7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.134522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.136221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.136221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.136394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] PATH_INFO: `/v3/projects/5e6c4cb5fc004864bab12042e42c62c6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.137683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6c4cb5fc004864bab12042e42c62c6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.140010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.244579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6c4cb5fc004864bab12042e42c62c6', 'request_id': 'req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:37.252538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6c4cb5fc004864bab12042e42c62c6', 'request_id': 'req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:37.267323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6c4cb5fc004864bab12042e42c62c6', 'request_id': 'req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:37.268877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:37.282936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] The token cache is being invalidate because project 5e6c4cb5fc004864bab12042e42c62c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:37.283248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:37.284264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-be7ac1b9-7ea3-4cfd-8aa5-50df1fcb3775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:37.285966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 937/1888] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:36 2022] DELETE /identity/v3/projects/5e6c4cb5fc004864bab12042e42c62c6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:37.292405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.292856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.293193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.316020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:37.317583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:37.397996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 952/1889] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:37.406819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.407108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.407391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.433983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:37.435263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:37.443535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 938/1890] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:37.448013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.463443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b270-3c6d-4646-90a9-61bc944bff2f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.469140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.469140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.469436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.469769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.470535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:37.470794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.472418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.474469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:37.495627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 953/1891] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:37.499822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.509742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289275be-7632-4920-baa4-33c5a66f4e7a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.514460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.515079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.515359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.515634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.516277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.518256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.546552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 939/1892] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:37 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:37.551221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.561330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bb43b-638b-4917-b879-7d325b361ad4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.620803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.621423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.621730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.622002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.622643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.624575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.648320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 954/1893] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:37 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:37.653868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.665183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cece2c9-ab97-4036-bdf8-a936d2695056 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.670927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.671563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.672185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.672453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.673100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.674735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.685101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 940/1894] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:37.689285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.700475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ed19f-28e1-4cf2-bd71-c79a97c90d92 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.705835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.706449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.706734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.707004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.707665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.709319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6827845-7047-4c1e-ace6-c7ce0a0d1dcf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.721347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 955/1895] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:37.725827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.736817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52f8ea-19b7-4557-a82d-73e36743c6aa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:37.742785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:37.743500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.743805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.744099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] PATH_INFO: `/v3/projects/0ec0887f55914b9690c176da11a0ef7a/users/0f33519d63a74f3e8ac3543af87dba49/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.761941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec0887f55914b9690c176da11a0ef7a, user_id=0f33519d63a74f3e8ac3543af87dba49, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:37.765794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:37.793764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 941/1896] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:37 2022] PUT /identity/v3/projects/0ec0887f55914b9690c176da11a0ef7a/users/0f33519d63a74f3e8ac3543af87dba49/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:37.797282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:37.797810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:37.798217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:37.837400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] MFA Rules not processed for user `0f33519d63a74f3e8ac3543af87dba49`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:37.838664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:37.926404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 956/1897] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:37.937978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:37.940592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:38.014064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cafee-3811-489b-a7ea-7be097357dee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:38.022284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:38.023019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.023298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.023573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:38.036711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:38.037762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425bd9e-b3ed-4d9d-bcc0-2fee1535414d service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:38.047241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 942/1898] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:38.072197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:38.136574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78cce62-b739-4d82-bd28-50b8556ec25a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:38.141097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:38.141759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.142035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.142300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] PATH_INFO: `/v3/users/0f33519d63a74f3e8ac3543af87dba49` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:38.143446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f33519d63a74f3e8ac3543af87dba49)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:38.145379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:38.148159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:38.198842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f33519d63a74f3e8ac3543af87dba49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.211019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f33519d63a74f3e8ac3543af87dba49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.226621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f33519d63a74f3e8ac3543af87dba49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.236570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f33519d63a74f3e8ac3543af87dba49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.237378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f33519d63a74f3e8ac3543af87dba49'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.244535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 957/1899] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:38 2022] DELETE /identity/v3/users/0f33519d63a74f3e8ac3543af87dba49 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:38.249395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.249762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.250055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:38.271149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:38.272370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:38.361740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 943/1900] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:38.371503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:38.373032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:38.440097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e352c17-2204-4c3a-9111-6066555761f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:38.447344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:38.448014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.448244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.448609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:38.459267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:38.460916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77ea24-91b3-40a1-940f-46fce29212fe service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:38.471149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 958/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:38.846791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:38.848999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:38.863457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533566a-1044-4608-a70e-d978e8dbff7e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:38.869553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:38.870339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.870580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.870853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] PATH_INFO: `/v3/projects/0ec0887f55914b9690c176da11a0ef7a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:38.872148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec0887f55914b9690c176da11a0ef7a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:38.874208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:38.916316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:38.942370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec0887f55914b9690c176da11a0ef7a', 'request_id': 'req-9ac3a264-71eb-496e-9a78-94d6e0aa26df'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.951031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec0887f55914b9690c176da11a0ef7a', 'request_id': 'req-9ac3a264-71eb-496e-9a78-94d6e0aa26df'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.960241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec0887f55914b9690c176da11a0ef7a', 'request_id': 'req-9ac3a264-71eb-496e-9a78-94d6e0aa26df'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.973025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] The token cache is being invalidate because project 0ec0887f55914b9690c176da11a0ef7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:38.973315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.974016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ac3a264-71eb-496e-9a78-94d6e0aa26df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:38.975733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 944/1902] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:38 2022] DELETE /identity/v3/projects/0ec0887f55914b9690c176da11a0ef7a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:38.981041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:38.981323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:38.981627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.006658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:39.008117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:39.084338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.084914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.084914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.090111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 959/1903] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:39.105443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:39.106875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:39.117451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 945/1904] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:39.122340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.132918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3c51e-a3c1-4d00-9991-1f8732a0b22d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.138400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.139012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.139280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.139542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.140191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:39.140504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.142159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.158551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 960/1905] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:39.160095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.170893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825353ab-337f-454f-a064-bdd5c1b09b8b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.176686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.177177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.177453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.177752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.178382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.180315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.222740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 946/1906] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:39 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:39.226528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.237520 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6791-44a1-4941-86a3-e17e3566f8b8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.248353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:39.312030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.312655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.312945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.313197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.313900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.315873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.341511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 961/1907] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:39 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:39.346310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.357002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f11a0-a2c6-40e8-8b21-ad6d04efa47d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.361933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.362535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.362821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.363071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.363734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.365330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.375295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 947/1908] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:39.379615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.390568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6a23e-31a0-47d4-81af-38e61469d5ad admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.398355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.398962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.399209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.399476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.400129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.402029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a0397-7948-4acc-bcc4-1b337a556fff admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.413680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 962/1909] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:39.418181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.428987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c69e90-a61e-4b90-be82-a2bf604053ab admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.434465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.435255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.435544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.435821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] PATH_INFO: `/v3/projects/997dd7a0c62947e09c6cab0cddcc75d5/users/7389adc9e92743ab921b375a2cbe117e/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.452057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997dd7a0c62947e09c6cab0cddcc75d5, user_id=7389adc9e92743ab921b375a2cbe117e, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.454787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.470418 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 948/1910] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:39 2022] PUT /identity/v3/projects/997dd7a0c62947e09c6cab0cddcc75d5/users/7389adc9e92743ab921b375a2cbe117e/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:39.480653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d934c01-3fbc-4cc5-aa8a-e066908ba071 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.481145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d934c01-3fbc-4cc5-aa8a-e066908ba071 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.481235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d934c01-3fbc-4cc5-aa8a-e066908ba071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.510985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3d934c01-3fbc-4cc5-aa8a-e066908ba071 None None] MFA Rules not processed for user `7389adc9e92743ab921b375a2cbe117e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:39.512310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d934c01-3fbc-4cc5-aa8a-e066908ba071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:39.601422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 963/1911] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:39.612457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:39.614239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:39.677179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00580bc-e93e-4657-aef7-edc90b4230d1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:39.680975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:39.681697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:39.681980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:39.682250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:39.691198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:39.692703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188ff39-49a9-4450-8713-13f433a54690 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:39.701980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 949/1912] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:40.748975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf38e-7e58-48e8-b2bc-c5ce1d5c3c2b None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:40.749508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf38e-7e58-48e8-b2bc-c5ce1d5c3c2b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:40.749675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf38e-7e58-48e8-b2bc-c5ce1d5c3c2b None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:40.751168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 964/1913] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:03:40.764519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:40.765016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:40.766983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:40.791125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:40.793094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:40.866513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:40.891757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 950/1914] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:03:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:41.124897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:41.125198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:41.125470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:41.146867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] MFA Rules not processed for user `7389adc9e92743ab921b375a2cbe117e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:41.148116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:41.156500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 965/1915] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:41.183333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:41.188630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:41.203321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110380be-f37f-4506-9ff2-4d3b340309cd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:41.209235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:41.209882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:41.210163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:41.210434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:41.219430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:41.221022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8036a9-43f6-4124-80c8-88d794dfca20 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:41.231252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 951/1916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:51.139616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:51.226979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=12 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:51.809516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:51.810962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:51.894611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11839e5b-d54b-464e-b389-23d97a20b485 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:51.900632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:51.901286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:51.901592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:51.901877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] PATH_INFO: `/v3/users/74ef516c242341cbb287dacc83f1b78c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:51.903112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ef516c242341cbb287dacc83f1b78c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:51.905070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:51.960677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ef516c242341cbb287dacc83f1b78c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:51.969280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ef516c242341cbb287dacc83f1b78c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:51.981579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ef516c242341cbb287dacc83f1b78c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:51.991438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ef516c242341cbb287dacc83f1b78c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:51.992013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ef516c242341cbb287dacc83f1b78c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:51.998263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 966/1917] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:51 2022] DELETE /identity/v3/users/74ef516c242341cbb287dacc83f1b78c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:52.002959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.003250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.003535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.028480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:52.029911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.092878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 952/1918] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:52.103527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:52.105476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.165773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2fa9e-664e-47b7-a852-570086f4505e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:52.169586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:52.170545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.170820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.171082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.181981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:52.183447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:52.193759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 967/1919] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:52.530503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:52.532060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.549746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf778840-1b42-4c69-8e95-120eb20a9d74 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:52.554466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:52.555220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.555664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.555766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] PATH_INFO: `/v3/users/02f6186669f74e85a51e53c58f45d9af` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.556908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f6186669f74e85a51e53c58f45d9af)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:52.558921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:52.617762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f6186669f74e85a51e53c58f45d9af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.618587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f6186669f74e85a51e53c58f45d9af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.619294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f6186669f74e85a51e53c58f45d9af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.629119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:52.630663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.638048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f6186669f74e85a51e53c58f45d9af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.645565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21378f7c-f8a0-4d12-9d4d-2fd2c2475540 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:52.650827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:52.651988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.652402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.652735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] PATH_INFO: `/v3/projects/897802aa26b441a896c49a5d1760f05f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.654203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897802aa26b441a896c49a5d1760f05f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:52.661335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e4b4a0cc-3252-4e3a-b242-2bde487d06fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f6186669f74e85a51e53c58f45d9af'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.674187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:52.705163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 953/1920] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:52 2022] DELETE /identity/v3/users/02f6186669f74e85a51e53c58f45d9af => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:52.710392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.710728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.711005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.712397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:52.748084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897802aa26b441a896c49a5d1760f05f', 'request_id': 'req-33ad6806-bb44-410b-8288-7fc3f32b7d28'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.748982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:52.751643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.761423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897802aa26b441a896c49a5d1760f05f', 'request_id': 'req-33ad6806-bb44-410b-8288-7fc3f32b7d28'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.771689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897802aa26b441a896c49a5d1760f05f', 'request_id': 'req-33ad6806-bb44-410b-8288-7fc3f32b7d28'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.790958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] The token cache is being invalidate because project 897802aa26b441a896c49a5d1760f05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:52.791406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.792399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.794205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 968/1921] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:52 2022] DELETE /identity/v3/projects/897802aa26b441a896c49a5d1760f05f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:52.800484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:52.801962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.835179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 954/1922] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:52.844523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:52.845896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:52.861395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:52.878199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6806-bb44-410b-8288-7fc3f32b7d28 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:52.883239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:52.883876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.884143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.884565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] PATH_INFO: `/v3/users/e8c671f1410f47c7bf7b944ebfdd303b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.885724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c671f1410f47c7bf7b944ebfdd303b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:52.887635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:52.905507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7a136-9948-409c-847e-7c2e49c03986 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:52.909264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:52.909919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:52.910195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:52.910472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:52.920933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:52.922872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:52.932797 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 955/1923] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:52.939404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c671f1410f47c7bf7b944ebfdd303b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.947874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c671f1410f47c7bf7b944ebfdd303b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.961122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c671f1410f47c7bf7b944ebfdd303b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.971137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c671f1410f47c7bf7b944ebfdd303b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.971842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c671f1410f47c7bf7b944ebfdd303b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:52.978401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 969/1924] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:52 2022] DELETE /identity/v3/users/e8c671f1410f47c7bf7b944ebfdd303b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:53.222239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:53.224096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:53.240054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b1ea2-be04-4061-80f0-96b5d8d3c65f service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:53.307677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:53.308351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.308628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.308901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] PATH_INFO: `/v3/projects/0ace49ae706e4cc4bfd0805b99f236f0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.310178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ace49ae706e4cc4bfd0805b99f236f0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:53.312091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:53.377081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ace49ae706e4cc4bfd0805b99f236f0', 'request_id': 'req-19a49e31-1c62-4fa1-9e70-e413882d1ba5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.383758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ace49ae706e4cc4bfd0805b99f236f0', 'request_id': 'req-19a49e31-1c62-4fa1-9e70-e413882d1ba5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.394741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ace49ae706e4cc4bfd0805b99f236f0', 'request_id': 'req-19a49e31-1c62-4fa1-9e70-e413882d1ba5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.417567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] The token cache is being invalidate because project 0ace49ae706e4cc4bfd0805b99f236f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:53.417567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.417992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.421098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 956/1925] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:53 2022] DELETE /identity/v3/projects/0ace49ae706e4cc4bfd0805b99f236f0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:53.424248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:53.425691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:53.439474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226dab2-df28-4d37-aa44-7d01be0bd14f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:53.459811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:53.479566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:53.481008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:53.508964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:53.509976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.510334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.510689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] PATH_INFO: `/v3/users/0e695d0d73a7489f89d5aafaf54bc5bf` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.512010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e695d0d73a7489f89d5aafaf54bc5bf)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:53.514291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:53.534570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:53.553475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a49e31-1c62-4fa1-9e70-e413882d1ba5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:53.560120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:53.561238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.561393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.561704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] PATH_INFO: `/v3/projects/25ae8a5188fc406fbb72749e8ff82699` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.563197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ae8a5188fc406fbb72749e8ff82699)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:53.565270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:53.596508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e695d0d73a7489f89d5aafaf54bc5bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.608735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e695d0d73a7489f89d5aafaf54bc5bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.626084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ae8a5188fc406fbb72749e8ff82699', 'request_id': 'req-ef031c2d-10ca-4600-ba40-14afb92d6a66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.628434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e695d0d73a7489f89d5aafaf54bc5bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.634904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ae8a5188fc406fbb72749e8ff82699', 'request_id': 'req-ef031c2d-10ca-4600-ba40-14afb92d6a66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.640677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e695d0d73a7489f89d5aafaf54bc5bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.641842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0bb514fb-6ac0-4c55-bcb2-d80edcb4a5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e695d0d73a7489f89d5aafaf54bc5bf'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.648776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ae8a5188fc406fbb72749e8ff82699', 'request_id': 'req-ef031c2d-10ca-4600-ba40-14afb92d6a66'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.651802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 970/1926] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:53 2022] DELETE /identity/v3/users/0e695d0d73a7489f89d5aafaf54bc5bf => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:53.661702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] The token cache is being invalidate because project 25ae8a5188fc406fbb72749e8ff82699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:53.662108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.662882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef031c2d-10ca-4600-ba40-14afb92d6a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:53.665589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 957/1927] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:53 2022] DELETE /identity/v3/projects/25ae8a5188fc406fbb72749e8ff82699 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:53.686964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.691337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.693250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.777010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:53.778429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:53.873978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 971/1928] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:53.880647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.881089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.881430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.905095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:53.906337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:53.918270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 958/1929] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:53.919925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:53.934927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a9fe7-4ced-449c-a408-0e453fd8787f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:53.941158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:53.941947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.942280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.942629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.943366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:53.943727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:53.945395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:53.961891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 972/1930] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:53.967471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:53.978896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a94c2d-79a1-4358-a416-69a88b86feb3 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:53.985048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:53.986102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:53.986378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:53.987021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:53.987570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:53.989430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.022598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 959/1931] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:53 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:54.028130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.054943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0df34-928d-44c0-a2eb-3b16de883452 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.110929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:54.153617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.155089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.155544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.155876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.156590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.159185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.191300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 973/1932] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:54.192620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.204487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5239ae-5059-4722-95d7-b8091eac9e38 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.211326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.212026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.212394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.212784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.213529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.215433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.229014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 960/1933] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:54.232823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.245352 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5948ec8-60f4-4c27-b27c-08e8331deddd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.256716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.257321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.257623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.257894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.258576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.260194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.274231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 974/1934] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:54.275435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.275863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.278026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:54.286428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ab4b1-4832-486c-b27b-c37d2d7c7b1b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.291544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503c484-4b70-4778-a3d9-5587e9255fc8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.294465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.295686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.296387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.297362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] PATH_INFO: `/v3/projects/4f46ff3df7734a4c98f6d22da4387eb7/users/1d472e507ad5407aad2d0ba9a4b31fe6/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.297491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.298388 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.298746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.299109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] PATH_INFO: `/v3/projects/6a9b9b66759840d18057c3c95c802177` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.300326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9b9b66759840d18057c3c95c802177)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.302688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.319817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f46ff3df7734a4c98f6d22da4387eb7, user_id=1d472e507ad5407aad2d0ba9a4b31fe6, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.323402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c8de3-d706-4d66-9a7d-b826c4719325 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.339637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 975/1935] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:54 2022] PUT /identity/v3/projects/4f46ff3df7734a4c98f6d22da4387eb7/users/1d472e507ad5407aad2d0ba9a4b31fe6/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:54.350105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.350388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.350657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.358831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9b9b66759840d18057c3c95c802177', 'request_id': 'req-d0518a81-ccad-4354-b883-b667d144294e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:54.364939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9b9b66759840d18057c3c95c802177', 'request_id': 'req-d0518a81-ccad-4354-b883-b667d144294e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:54.376473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9b9b66759840d18057c3c95c802177', 'request_id': 'req-d0518a81-ccad-4354-b883-b667d144294e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:54.392016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] The token cache is being invalidate because project 6a9b9b66759840d18057c3c95c802177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:54.392313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:54.393331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d0518a81-ccad-4354-b883-b667d144294e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:54.395976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] MFA Rules not processed for user `1d472e507ad5407aad2d0ba9a4b31fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:54.397468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:54.398654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 961/1936] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:54 2022] DELETE /identity/v3/projects/6a9b9b66759840d18057c3c95c802177 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:54.400075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f356bb-08c6-49f8-abd4-f73b14da907f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.400361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f356bb-08c6-49f8-abd4-f73b14da907f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.400637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f356bb-08c6-49f8-abd4-f73b14da907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.424795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c1f356bb-08c6-49f8-abd4-f73b14da907f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:54.426064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c1f356bb-08c6-49f8-abd4-f73b14da907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:54.442072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:54.519601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 962/1937] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:54.527543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.528088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.528506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.560382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:54.561726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:54.570593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 963/1938] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:54.575115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.586105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df370eb-66fe-4eef-aca4-241bd0ed6cad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.588193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 976/1939] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:54.591715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.592506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.593234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.593653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.594211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.594531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:54.594934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.595698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:54.597036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.611367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 964/1940] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:03:54.616424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.618480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a382218-006e-4c26-9849-082eb7539ac4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.624969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.625825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.626112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.626381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.627221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.629256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.630947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25713dc-c6b6-4cb0-962e-08de6a23e1d9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.638346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.639502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.639993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.640466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.641976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.644895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.665501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 977/1941] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:54.670110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.687028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 965/1942] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:54.687739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.744323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:54.765373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0e2c-c272-45f2-bcc6-9825f17ed9d9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.768106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d96bc4-2154-4963-a806-380c4accea8f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.774213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.774397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.774845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.775025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.775133 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.775304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.775394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.775572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.776000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.776214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.777951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.778212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.805403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 978/1943] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:54.809656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.809796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 966/1944] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:54 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:54.814538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.820014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ccebf-a797-4977-b3af-036badb2a20d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.825045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.825889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.827700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45cee71-01b6-4c03-bb3b-bb891d141503 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.831778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.832142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.832808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.832953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.833557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.833909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.834380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.834564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.835899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.837103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.845889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 979/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:54.850158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.853382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 967/1946] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:54.857081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.862204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d8dd3-e531-43d2-9b2d-2812f066dd6c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.867472 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.867768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0748f-0f57-4aaf-a09a-acced7f0748a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.868546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.868922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.869277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] PATH_INFO: `/v3/projects/efac9c4aabc5472d8984c3359dfac6b3/users/471176f694fd4f9ca72c631dd61be80d/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.872467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:54.873081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:54.873361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:54.873665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:54.874515 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.876145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.891521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efac9c4aabc5472d8984c3359dfac6b3, user_id=471176f694fd4f9ca72c631dd61be80d, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:54.892737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 968/1947] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:54.895223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:54.906632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.915549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 980/1948] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:54 2022] PUT /identity/v3/projects/efac9c4aabc5472d8984c3359dfac6b3/users/471176f694fd4f9ca72c631dd61be80d/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:54.921776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b0228-27ae-41c1-a0f4-6ad0b6f0fbee admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:54.922924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:54.980521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:55.000552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.001235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.001543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.001838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] PATH_INFO: `/v3/projects/1e1a085fa5154c658a5b69e9dc85ce37/users/050326a57dba45d8bc55dabac8d8b039/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.007753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82b244-68c4-43ef-8e86-7f2cf8c1e7f5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.013133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.013781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.014056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.014323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.014967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.016589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.018259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1a085fa5154c658a5b69e9dc85ce37, user_id=050326a57dba45d8bc55dabac8d8b039, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.021036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d47881-40f4-479d-9f1d-944e68731813 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.026751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 981/1949] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:03:55.031198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.035739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 969/1950] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:54 2022] PUT /identity/v3/projects/1e1a085fa5154c658a5b69e9dc85ce37/users/050326a57dba45d8bc55dabac8d8b039/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:55.040738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.041023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.041295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.041465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f18ca-03d5-418b-8499-6908454de039 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.059728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:55.060958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:55.115268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.115947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.116414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.116698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] PATH_INFO: `/v3/projects/efac9c4aabc5472d8984c3359dfac6b3/users/471176f694fd4f9ca72c631dd61be80d/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.118764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efac9c4aabc5472d8984c3359dfac6b3, user_id=471176f694fd4f9ca72c631dd61be80d, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.121682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc8752-3a6c-475f-aef2-0c9f5ba630d9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.134507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 982/1951] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:55 2022] PUT /identity/v3/projects/efac9c4aabc5472d8984c3359dfac6b3/users/471176f694fd4f9ca72c631dd61be80d/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:55.140646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 970/1952] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:55.145828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.146118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.146392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.151351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.153643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:55.175631 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] MFA Rules not processed for user `471176f694fd4f9ca72c631dd61be80d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:55.177232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:55.209557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:55.223658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd925fb-6ecf-4c97-bf09-dca27cd1c351 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.262246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 983/1953] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:55.275424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.277252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:55.284713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.285429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.285746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.286159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.356494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:55.366321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032eeb8a-aec3-42cb-b362-93be320138b1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.370095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.370825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.371214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.371543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.381020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.382373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.382685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.384160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.393271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 984/1954] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:55.404775 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 971/1955] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:55.771716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.782783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73d0b-459b-4507-b969-7ad8447c35c3 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.787397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471176f694fd4f9ca72c631dd61be80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efac9c4aabc5472d8984c3359dfac6b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.788004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.788435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.788709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.789364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:55.789702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.791668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.807985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 985/1956] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:55.809396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.823843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6b57-151f-4e0e-8df6-5a44c481b566 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.828810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471176f694fd4f9ca72c631dd61be80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efac9c4aabc5472d8984c3359dfac6b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.829601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.829937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.830221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.830915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.833043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.861927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 972/1957] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:55 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:55.866234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:55.876770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db0797-fef7-449a-9346-bb121bf92636 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:55.950633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471176f694fd4f9ca72c631dd61be80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efac9c4aabc5472d8984c3359dfac6b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:55.951372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:55.951856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:55.953280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:55.953280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:55.953280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:55.955325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f0932-ccf7-4a3f-b7de-5fb6ab966ee4 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:55.973869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 986/1958] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:03:56.008066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.018599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a953b65-b030-4459-9223-941710def6ae tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.023148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471176f694fd4f9ca72c631dd61be80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'efac9c4aabc5472d8984c3359dfac6b3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.023790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.024072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.024348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] PATH_INFO: `/v3/projects/8500a245165846148e279985b22aeb48` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.025470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8500a245165846148e279985b22aeb48)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.027570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.080676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8500a245165846148e279985b22aeb48', 'request_id': 'req-c7f4adf8-3856-4825-a78c-ae35abd73a87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.087476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8500a245165846148e279985b22aeb48', 'request_id': 'req-c7f4adf8-3856-4825-a78c-ae35abd73a87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.096609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8500a245165846148e279985b22aeb48', 'request_id': 'req-c7f4adf8-3856-4825-a78c-ae35abd73a87'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.109503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] The token cache is being invalidate because project 8500a245165846148e279985b22aeb48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:56.109960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.110735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.112464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 973/1959] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:56 2022] DELETE /identity/v3/projects/8500a245165846148e279985b22aeb48 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:56.254257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.254548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.254824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.305793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] MFA Rules not processed for user `050326a57dba45d8bc55dabac8d8b039`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:56.307107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.346245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:56.385448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.388867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.431240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 987/1960] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:56.435708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.437257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.467027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f4adf8-3856-4825-a78c-ae35abd73a87 tempest-VolumeQuotaClassesTest-520155022 tempest-VolumeQuotaClassesTest-520155022-project-admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.471989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.472611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.472841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.473111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] PATH_INFO: `/v3/users/1d472e507ad5407aad2d0ba9a4b31fe6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.474271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d472e507ad5407aad2d0ba9a4b31fe6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.476191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.525272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476bad-d0c5-4db0-9d41-d8f43eaff8ca None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.530636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.531283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.531581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.531989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.532705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.534853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.540173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:56.545652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d472e507ad5407aad2d0ba9a4b31fe6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.546677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d472e507ad5407aad2d0ba9a4b31fe6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.547891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d472e507ad5407aad2d0ba9a4b31fe6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.592057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d472e507ad5407aad2d0ba9a4b31fe6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.607431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-476ad864-6286-4d74-82c8-82dd6a9683ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d472e507ad5407aad2d0ba9a4b31fe6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:56.616318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 988/1961] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:56 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:56.617438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.619628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.653918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 974/1962] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:56 2022] DELETE /identity/v3/users/1d472e507ad5407aad2d0ba9a4b31fe6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:56.658298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.658645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.658931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.679601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:56.680929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.717931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb213c-d1a2-40fe-acdd-9c23f75bedb5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.754048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 975/1963] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:03:56.766696 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.768281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:56.778074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.778695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.779188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.779292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.779896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.781905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.800607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:56.808983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 989/1964] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:56 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:56.811353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.822614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aac721-7af4-476a-b0ff-9089ef013f96 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.827429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.827592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e42c1-2fcc-4351-9016-7f3026b80ed4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.828045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.828294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.828566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.829231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.831158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.831327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.831940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.832223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.832498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.841495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 990/1965] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:56.841634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.843036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.845606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.852484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 976/1966] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:03:56.855073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a3f15-5fce-4568-aa68-91e7c23e9e8e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.859992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.860679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.860956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.861229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] PATH_INFO: `/v3/projects/27fdf600d1d548e0a51256e12b8b932e/users/e43b139327834956b755a04b454f2df6/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.876538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fdf600d1d548e0a51256e12b8b932e, user_id=e43b139327834956b755a04b454f2df6, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.879335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.897129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 991/1967] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:56 2022] PUT /identity/v3/projects/27fdf600d1d548e0a51256e12b8b932e/users/e43b139327834956b755a04b454f2df6/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:56.901240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:56.911617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8e4fa-049c-4d38-85a9-72b112e9c4fb service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:56.970758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:56.971432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:56.971728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:56.972006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:56.972662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:56.974332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:56.975708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:56.986002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 977/1968] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:56.990402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:57.002050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb88b1-d781-4c39-9771-2ac09b04052a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:57.008035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:57.008674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:57.008951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:57.009221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] PATH_INFO: `/v3/projects/27fdf600d1d548e0a51256e12b8b932e/users/e43b139327834956b755a04b454f2df6/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:57.011201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fdf600d1d548e0a51256e12b8b932e, user_id=e43b139327834956b755a04b454f2df6, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:57.014103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:57.024986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 992/1969] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:56 2022] PUT /identity/v3/projects/27fdf600d1d548e0a51256e12b8b932e/users/e43b139327834956b755a04b454f2df6/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:57.130759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:57.132718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:57.165780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb1eb-5da5-4ad3-aa46-9c583da19f9a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:57.245770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:57.246362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:57.246638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:57.247345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] PATH_INFO: `/v3/projects/4f46ff3df7734a4c98f6d22da4387eb7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:57.248093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f46ff3df7734a4c98f6d22da4387eb7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:57.250151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:57.308822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f46ff3df7734a4c98f6d22da4387eb7', 'request_id': 'req-99eacd65-d031-4590-8521-a117d7f8c23f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.315668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f46ff3df7734a4c98f6d22da4387eb7', 'request_id': 'req-99eacd65-d031-4590-8521-a117d7f8c23f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.324812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f46ff3df7734a4c98f6d22da4387eb7', 'request_id': 'req-99eacd65-d031-4590-8521-a117d7f8c23f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.342259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] The token cache is being invalidate because project 4f46ff3df7734a4c98f6d22da4387eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:57.342956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.344214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99eacd65-d031-4590-8521-a117d7f8c23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.346585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 978/1970] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:57 2022] DELETE /identity/v3/projects/4f46ff3df7734a4c98f6d22da4387eb7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:57.350604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:57.352750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:57.367705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c103bc-a31b-436e-89c3-82b55b2e2db3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:57.498301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:57.499495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:57.499954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:57.500316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] PATH_INFO: `/v3/users/471176f694fd4f9ca72c631dd61be80d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:57.502623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471176f694fd4f9ca72c631dd61be80d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:57.507039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:57.580656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471176f694fd4f9ca72c631dd61be80d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.590404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471176f694fd4f9ca72c631dd61be80d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.606946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471176f694fd4f9ca72c631dd61be80d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.620361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471176f694fd4f9ca72c631dd61be80d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.621273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471176f694fd4f9ca72c631dd61be80d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:57.634775 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 993/1971] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:57 2022] DELETE /identity/v3/users/471176f694fd4f9ca72c631dd61be80d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:58.149794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.150286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.150366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.200956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] MFA Rules not processed for user `e43b139327834956b755a04b454f2df6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:58.204222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.329620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 979/1972] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:58.344144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:58.346125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.357570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:58.359513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.381155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:58.390693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:58.423742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88194ccc-b99b-4d61-a63b-be1261c8b130 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:58.428966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:58.429763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.430410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.430521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.432890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.457496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8815f-ccb9-48ad-ad2e-e0637c5181a5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:58.464684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:58.465792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.466282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.466615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] PATH_INFO: `/v3/projects/efac9c4aabc5472d8984c3359dfac6b3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.481215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=efac9c4aabc5472d8984c3359dfac6b3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:58.486058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:58.521957 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:58.524810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa22aae-33d4-46dc-9bda-a50f7c7399ef service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:58.539318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 994/1973] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:03:58.630455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efac9c4aabc5472d8984c3359dfac6b3', 'request_id': 'req-22ce715e-af03-40dc-8139-2d2ca750fafe'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:58.639772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efac9c4aabc5472d8984c3359dfac6b3', 'request_id': 'req-22ce715e-af03-40dc-8139-2d2ca750fafe'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:58.671374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efac9c4aabc5472d8984c3359dfac6b3', 'request_id': 'req-22ce715e-af03-40dc-8139-2d2ca750fafe'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:58.693571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] The token cache is being invalidate because project efac9c4aabc5472d8984c3359dfac6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:03:58.693881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:58.694565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-22ce715e-af03-40dc-8139-2d2ca750fafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:58.697970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 980/1974] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:03:58 2022] DELETE /identity/v3/projects/efac9c4aabc5472d8984c3359dfac6b3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:03:58.703455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.703982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.704450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.738313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:58.741359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.831976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 995/1975] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:58.841209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.841974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.841974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.895778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:58.899514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:58.927978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 981/1976] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:58.929670 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:58.967781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a865c4-2014-4443-bf3e-2d82384e16af None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:58.991146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:58.992643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:58.993960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:58.996443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:58.997159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:03:58.997528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.010993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.053585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 996/1977] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:03:59.054561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.086513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a60bf-abd4-4611-add4-b7fe6b409d60 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.100864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.101662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.102105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.102367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.103178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.105890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.144861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 982/1978] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:03:59.146451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.161668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd63e3-a890-4820-8769-b7ab5a65f0a7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.242363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.243565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.243565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.243762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.244405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.247898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.276195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:59.286760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 997/1979] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:59.290824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.309673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf82f12c-a435-46ef-b36f-3a7351f99b60 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.318231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.319705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.320967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.320967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.321270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.326160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.328133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.330562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:59.347995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a398f0-636e-47ce-8970-0765311962f2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.356915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 983/1980] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:59.357097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.357097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.357097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.357435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] PATH_INFO: `/v3/users/dd1bff35591447db814f07bbbbad7e3d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.358077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.358881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd1bff35591447db814f07bbbbad7e3d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.362677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.370593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf12f2a-49c4-4584-bae7-ab638d1ce8ec admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.389079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.390080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.390443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.390879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.391544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.393225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.409719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 984/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:03:59.411180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.425373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6253b-f59f-463a-aa60-2d086acaa384 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.431568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.432372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.432714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.433124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] PATH_INFO: `/v3/projects/a2cdd9ee98b2434b9baac6f2339d7f9d/users/779b87e8945446f586b3e10095e485d1/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.452856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1bff35591447db814f07bbbbad7e3d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:59.458034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdd9ee98b2434b9baac6f2339d7f9d, user_id=779b87e8945446f586b3e10095e485d1, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.460669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6783a-1490-4243-b8ef-3000cc0cc782 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.493831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1bff35591447db814f07bbbbad7e3d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:59.511222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 985/1982] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:03:59 2022] PUT /identity/v3/projects/a2cdd9ee98b2434b9baac6f2339d7f9d/users/779b87e8945446f586b3e10095e485d1/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:59.523655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.524537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.525177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.525866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1bff35591447db814f07bbbbad7e3d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:59.544686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1bff35591447db814f07bbbbad7e3d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:59.545646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a1faff59-bdc4-4981-829c-c55a122d3353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1bff35591447db814f07bbbbad7e3d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:03:59.563060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 998/1983] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:03:59 2022] DELETE /identity/v3/users/dd1bff35591447db814f07bbbbad7e3d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:03:59.564776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.565618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.565994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.605574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] MFA Rules not processed for user `779b87e8945446f586b3e10095e485d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:59.608784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:59.615557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:03:59.617132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:59.641832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:59.732318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 986/1984] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:59.744887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.744887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:59.749434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 999/1985] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:03:59.764698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8a00a-e8fc-4ca6-9f6a-4a3886320c9f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.766933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.768582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:03:59.781027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.783038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.784073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.785960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.786849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.790736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.860830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 987/1986] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:03:59.865404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:03:59.893208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adae69a-6723-4832-8030-3ab4e6caf9b9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.931310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:03:59.966408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5cd4f3-cb70-46f6-b1ed-e9e9c28e6998 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:03:59.970430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.971259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.971540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.971810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.972887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:03:59.973769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:03:59.974252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:03:59.974717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:03:59.975582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.977982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:03:59.988142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:03:59.990787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.010265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 988/1987] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:03:59 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:00.010471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1000/1988] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:00.015011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.026767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49c6f-a52d-4686-bcca-ee21132d6ce5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.032006 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.032725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.033067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.033490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.034278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.036082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.048142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 989/1989] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:00.052376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.062345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1df75-5417-497e-ab94-ae9d6143578c service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.067529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.068198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.068582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.068927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.069921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.071587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.089456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.095528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1001/1990] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:00.104168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b984b-1948-49fa-80d8-1418bdf62d29 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.110493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.111256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.111653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.112112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] PATH_INFO: `/v3/projects/020748e4f9de46b3a68b3dd56921fb4c/users/3efbc5d1508844c59e83d1ab134c5597/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.133453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020748e4f9de46b3a68b3dd56921fb4c, user_id=3efbc5d1508844c59e83d1ab134c5597, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.137817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1cab7-2fb9-4e69-a27a-cf9aea33f296 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.164988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.164988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.166571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 990/1991] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:00 2022] PUT /identity/v3/projects/020748e4f9de46b3a68b3dd56921fb4c/users/3efbc5d1508844c59e83d1ab134c5597/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:00.170335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.170907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.171403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.226273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] MFA Rules not processed for user `3efbc5d1508844c59e83d1ab134c5597`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:00.228805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.268850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3945392-9c6b-4316-9541-a4952d790bc9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.292864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:00.332442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.332667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.332948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.333219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.335581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.343383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 991/1992] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:00.348362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.417968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.420661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.440454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1002/1993] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:04:00.455560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.457944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.476850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562178d-d2ee-4105-adbe-1c809284bb17 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.492297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72e02-ca43-4482-ab50-d7d41899ed9a service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.506876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.507552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.508258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.508542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] PATH_INFO: `/v3/projects/12a4bd2c837d4d00a0dadec048e04fde` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.511097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a4bd2c837d4d00a0dadec048e04fde)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.513044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.515715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.516463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.516813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.517176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.526441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.532677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.572960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:00.612793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 992/1994] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:00.618350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.630113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a4bd2c837d4d00a0dadec048e04fde', 'request_id': 'req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:00.645434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a4bd2c837d4d00a0dadec048e04fde', 'request_id': 'req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:00.663648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a4bd2c837d4d00a0dadec048e04fde', 'request_id': 'req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:00.691575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] The token cache is being invalidate because project 12a4bd2c837d4d00a0dadec048e04fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:00.691891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:00.693183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a17fdd15-64c0-40a9-a62b-0ee0c29a246b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:00.700656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1003/1995] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:00 2022] DELETE /identity/v3/projects/12a4bd2c837d4d00a0dadec048e04fde => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:00.702460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb0ab-9511-4fe8-befb-256dcde1de0c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.702748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb0ab-9511-4fe8-befb-256dcde1de0c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.703356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacb0ab-9511-4fe8-befb-256dcde1de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.734778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362d747-8552-4822-950d-6970e213cae9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.742649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aaacb0ab-9511-4fe8-befb-256dcde1de0c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:00.744725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aaacb0ab-9511-4fe8-befb-256dcde1de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.745525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.746578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.750703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.751127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.752679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.755117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.766170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1004/1996] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:00.775853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.776140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.776410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.808341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 993/1997] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:00.808582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:00.809121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.810041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.812947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:00.824459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1005/1998] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:00.825485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.836715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc31e-74c6-4d2f-840f-37e18f7dfcac None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.841838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.842456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.842730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.842996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.844138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:00.844138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.845686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.858471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1006/1999] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:00.863329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.866909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7295c0-dc6e-419f-b7b4-08e22022ad3b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.874881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16aae-f662-497b-a62d-4956e5e63beb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.880395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.880563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.881015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.881120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.881289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.881386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.881586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.881713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.882222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.882341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:00.884138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.884994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:00.911605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 994/2000] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:04:00.913452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:00.948655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef715d2-62b0-470c-b0ae-79732bd9561f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:00.970126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1007/2001] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:04:00.970420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:00.971183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:00.971459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:00.971727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] PATH_INFO: `/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb/users/bc2b69b177be4e24ad7db32db87379c5/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:00.972153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.016912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c5d404194048a795b6e8b3ddf2f0fb, user_id=bc2b69b177be4e24ad7db32db87379c5, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.023301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.045166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 995/2002] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:00 2022] PUT /identity/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb/users/bc2b69b177be4e24ad7db32db87379c5/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:01.050025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.074166 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:01.079956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f26039-9b00-4bae-b5d5-5988f310b4fd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.137315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eece5-a7c8-4778-a57a-f69f6622cb60 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.144540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.145193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.145713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.146000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.146711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.148340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.172260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.175745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 996/2003] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:01.175935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.175935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.175935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.175935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.176435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.177361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.190035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2aaf89-d509-473e-b09d-5d4cd13d5608 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.195760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110f074-5587-485f-9721-439f21902196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.195942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f074-5587-485f-9721-439f21902196 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.196175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f074-5587-485f-9721-439f21902196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.196433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f074-5587-485f-9721-439f21902196 admin admin] PATH_INFO: `/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb/users/bc2b69b177be4e24ad7db32db87379c5/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.198640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110f074-5587-485f-9721-439f21902196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c5d404194048a795b6e8b3ddf2f0fb, user_id=bc2b69b177be4e24ad7db32db87379c5, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.201635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110f074-5587-485f-9721-439f21902196 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.220103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 997/2004] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:01 2022] PUT /identity/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb/users/bc2b69b177be4e24ad7db32db87379c5/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:01.229826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:01.230923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.231396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.231682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.242321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1008/2005] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:00 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:01.243429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.286284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d4c52-46b5-4e53-ad8b-825dad6e48f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.300583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] MFA Rules not processed for user `bc2b69b177be4e24ad7db32db87379c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:01.301955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:01.397676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.400451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.400816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.402610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.403320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.408898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 998/2006] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:01.409045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.417649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.420861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1009/2007] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:01.422233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.423336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:01.436263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4562-fb3c-498f-b8f1-7cbd04ef9074 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.449052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.449733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.450010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.450281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.450934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.452966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.468642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1010/2008] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:01.469899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.481596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1f03b-1172-4f36-ac41-408126233827 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.485807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.486611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.486893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.487614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] PATH_INFO: `/v3/projects/4819532e3c014ac8acc8a600402826d9/users/aebc42d94b3e4d6fb3db6a3586c2ec96/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.509313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a843f-f2ee-4c9d-94ba-346a538ad339 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.509581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4819532e3c014ac8acc8a600402826d9, user_id=aebc42d94b3e4d6fb3db6a3586c2ec96, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.512312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870bae-c884-4ac5-8cb6-a3017b5850e5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.514561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.515188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.515463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.515734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.525968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.527403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.527511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1011/2009] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:01 2022] PUT /identity/v3/projects/4819532e3c014ac8acc8a600402826d9/users/aebc42d94b3e4d6fb3db6a3586c2ec96/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:01.539252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 999/2010] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:01.543901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.544186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.544469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.578138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] MFA Rules not processed for user `aebc42d94b3e4d6fb3db6a3586c2ec96`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:01.579382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:01.663567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1012/2011] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:01.675120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:01.676560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:01.687860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f1f93-53fe-47b5-b802-f40aa7ac51f7 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:01.715600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:01.744839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:01.745746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:01.746075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:01.746433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:01.756907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:01.758619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:01.769561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1000/2012] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:02.320134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:02.351621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77a566-c119-4b24-ab93-a42fd1ac0834 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:02.366009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:02.366930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:02.367407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:02.367849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:02.379136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:02.460443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:02.480543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:02.482174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d315329-0d17-4526-a439-d7da3b6bc799 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:02.492903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1013/2013] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:05.291141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:05.378567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bad11-c354-45df-a8a1-cbbfe61772fc service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:05.389794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:05.390865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:05.391234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:05.391657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] PATH_INFO: `/v3/users/aebc42d94b3e4d6fb3db6a3586c2ec96` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:05.393435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebc42d94b3e4d6fb3db6a3586c2ec96)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:05.396952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:05.489715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebc42d94b3e4d6fb3db6a3586c2ec96'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:05.490632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebc42d94b3e4d6fb3db6a3586c2ec96'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:05.491876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebc42d94b3e4d6fb3db6a3586c2ec96'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:05.509904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebc42d94b3e4d6fb3db6a3586c2ec96'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:05.519763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebc42d94b3e4d6fb3db6a3586c2ec96'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:05.543783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1001/2014] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:05 2022] DELETE /identity/v3/users/aebc42d94b3e4d6fb3db6a3586c2ec96 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:05.545655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:05.546146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:05.547403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:05.568792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:05.570097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:05.655222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1014/2015] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:05.665733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:05.667668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:05.745117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b79580-2d3d-41a5-a44c-7f3d5e88ab5e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:05.750940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:05.751875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:05.752358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:05.752965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:05.764301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:05.770262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:05.772658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c9368-f6cb-4fc0-b0ff-d7cc044f7cd6 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:05.802643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1002/2016] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:04:06.161826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.163693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:06.177184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8032483-224f-49aa-9f93-9f7983d79040 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.182836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.183778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.184049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.184449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] PATH_INFO: `/v3/projects/4819532e3c014ac8acc8a600402826d9` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.185451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4819532e3c014ac8acc8a600402826d9)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.187645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.237930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4819532e3c014ac8acc8a600402826d9', 'request_id': 'req-a2e1df98-e738-421f-962c-e502568890c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:06.243458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4819532e3c014ac8acc8a600402826d9', 'request_id': 'req-a2e1df98-e738-421f-962c-e502568890c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:06.249875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4819532e3c014ac8acc8a600402826d9', 'request_id': 'req-a2e1df98-e738-421f-962c-e502568890c4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:06.265196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] The token cache is being invalidate because project 4819532e3c014ac8acc8a600402826d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:06.265487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:06.266195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a2e1df98-e738-421f-962c-e502568890c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:06.267795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1015/2017] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:06 2022] DELETE /identity/v3/projects/4819532e3c014ac8acc8a600402826d9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:06.272853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953240-e362-4ed5-b663-f7712531d069 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.273267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953240-e362-4ed5-b663-f7712531d069 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.273620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40953240-e362-4ed5-b663-f7712531d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.293949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-40953240-e362-4ed5-b663-f7712531d069 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:06.295165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-40953240-e362-4ed5-b663-f7712531d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:06.360227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1003/2018] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:06.374748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.375197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.375633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.398573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:06.399783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:06.408315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1016/2019] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:06.412800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40953240-e362-4ed5-b663-f7712531d069 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.422861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40953240-e362-4ed5-b663-f7712531d069 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.427792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.428398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.428676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.428977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.429657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:06.429943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.432099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.450244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1004/2020] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:06.454562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.464823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47cc87-2f88-4615-9cf0-f2abaa43209e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.470228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.470858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.471272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.471549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.472229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.474199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.509287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1017/2021] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:06 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:06.510588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.522539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f1d58-5ca6-4b64-aa8e-609a47e9df78 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.588866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.589858 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.590141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.590415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.591060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.592981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.616215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1005/2022] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:06 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:06.620417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.630970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a495a-8a22-437d-a560-d68884c4453a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.635743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.636445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.636785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.637117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.637885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.639536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.650124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1018/2023] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:06.654201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.664589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4889d-d9fa-44c5-b040-3be695d48b0b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.670729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.671339 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.671612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.671879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.672988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.674675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770093a4-61a0-4f70-8a67-ed3d2aa22f10 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.686224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1006/2024] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:06.690767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.701246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26efa86-c344-417d-8a6f-1f72746c03ca admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.706082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.706762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.707033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.707324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] PATH_INFO: `/v3/projects/609a52d56a4f4aaeb50a3790f4f1edc4/users/5b97116addc342b0828567c3472089ca/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.722809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609a52d56a4f4aaeb50a3790f4f1edc4, user_id=5b97116addc342b0828567c3472089ca, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.725483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.740583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1019/2025] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:06 2022] PUT /identity/v3/projects/609a52d56a4f4aaeb50a3790f4f1edc4/users/5b97116addc342b0828567c3472089ca/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:06.751370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.751655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.751945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.780632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] MFA Rules not processed for user `5b97116addc342b0828567c3472089ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:06.781986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:06.830967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:06.850734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1007/2026] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:06.865527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:06.867425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:06.877721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:06.932821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01aa73a-b140-4398-821b-15727ce34b43 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:06.936985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:06.937641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:06.937920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:06.938190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:06.947762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:06.949424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:06.958690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1020/2027] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:07.088530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:07.090160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:07.156706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9692264-8b26-4cbb-99be-dfdb74483292 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:07.161279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:07.161936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:07.162211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:07.162483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:07.164300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:07.228621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:07.230266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f77e2-ab85-4d61-83f2-29ddb924851f service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:07.239918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1008/2028] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:09.984766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:09.986448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:10.056902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684cfb5-ff11-4b28-b012-dd0be37de691 service glance] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:10.061459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:10.062126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:10.062395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:10.062659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] PATH_INFO: `/v3/users/779b87e8945446f586b3e10095e485d1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:10.063894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=779b87e8945446f586b3e10095e485d1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:10.065941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:10.119812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779b87e8945446f586b3e10095e485d1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:10.127846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779b87e8945446f586b3e10095e485d1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:10.140145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779b87e8945446f586b3e10095e485d1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:10.149248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779b87e8945446f586b3e10095e485d1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:10.150148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779b87e8945446f586b3e10095e485d1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:10.162773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1021/2029] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:09 2022] DELETE /identity/v3/users/779b87e8945446f586b3e10095e485d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:10.164731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:10.165151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:10.165458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:10.186394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:10.187631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:10.283212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1009/2030] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:10.293678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:10.295327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:10.398015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:10.433813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f3140-e522-4beb-a0ec-33f2928d9a8a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:10.438647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:10.439316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:10.439559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:10.439827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:10.468487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:10.470189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:10.487856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1022/2031] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:11.061415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:11.063934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:11.083773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161c94e-bad8-4c16-b931-e311d5e6450b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:11.088528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:11.089169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:11.089497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:11.089816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] PATH_INFO: `/v3/projects/a2cdd9ee98b2434b9baac6f2339d7f9d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:11.090976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cdd9ee98b2434b9baac6f2339d7f9d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:11.093179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:11.164034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:11.180946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cdd9ee98b2434b9baac6f2339d7f9d', 'request_id': 'req-20572762-df7b-480b-883b-1a250ce02593'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.193329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cdd9ee98b2434b9baac6f2339d7f9d', 'request_id': 'req-20572762-df7b-480b-883b-1a250ce02593'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.203528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cdd9ee98b2434b9baac6f2339d7f9d', 'request_id': 'req-20572762-df7b-480b-883b-1a250ce02593'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.219368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] The token cache is being invalidate because project a2cdd9ee98b2434b9baac6f2339d7f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:11.219733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.220420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.222437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1010/2032] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:11 2022] DELETE /identity/v3/projects/a2cdd9ee98b2434b9baac6f2339d7f9d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:11.226793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:11.236994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4c61a-76c8-4d77-b24d-c3e02abf4ae8 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:11.318156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:11.318882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:11.319209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:11.319537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] PATH_INFO: `/v3/users/3efbc5d1508844c59e83d1ab134c5597` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:11.320996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efbc5d1508844c59e83d1ab134c5597)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:11.323084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:11.374711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efbc5d1508844c59e83d1ab134c5597'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.383338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efbc5d1508844c59e83d1ab134c5597'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.400834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efbc5d1508844c59e83d1ab134c5597'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.413297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efbc5d1508844c59e83d1ab134c5597'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.414105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efbc5d1508844c59e83d1ab134c5597'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.432710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1023/2033] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:11 2022] DELETE /identity/v3/users/3efbc5d1508844c59e83d1ab134c5597 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:11.809903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:11.811350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:11.829103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20572762-df7b-480b-883b-1a250ce02593 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:11.894100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:11.894788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:11.895060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:11.895328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] PATH_INFO: `/v3/projects/020748e4f9de46b3a68b3dd56921fb4c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:11.896461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=020748e4f9de46b3a68b3dd56921fb4c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:11.898719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:11.949406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020748e4f9de46b3a68b3dd56921fb4c', 'request_id': 'req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.955393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020748e4f9de46b3a68b3dd56921fb4c', 'request_id': 'req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.966948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020748e4f9de46b3a68b3dd56921fb4c', 'request_id': 'req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.979124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] The token cache is being invalidate because project 020748e4f9de46b3a68b3dd56921fb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:11.979414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.980048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:11.981580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1011/2034] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:11 2022] DELETE /identity/v3/projects/020748e4f9de46b3a68b3dd56921fb4c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:11.985506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:11.986932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:11.999487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295f24b-826f-476c-a956-226a0ead6b68 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:12.048009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:12.059463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:12.060137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.060505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.060781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] PATH_INFO: `/v3/users/bc2b69b177be4e24ad7db32db87379c5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.062196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2b69b177be4e24ad7db32db87379c5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:12.064560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:12.115331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2b69b177be4e24ad7db32db87379c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.120934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2b69b177be4e24ad7db32db87379c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.133962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2b69b177be4e24ad7db32db87379c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.144734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2b69b177be4e24ad7db32db87379c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.145389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-0ac2743d-c91f-474d-8ebc-fafe4597d1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2b69b177be4e24ad7db32db87379c5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.151488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1024/2035] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:11 2022] DELETE /identity/v3/users/bc2b69b177be4e24ad7db32db87379c5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:12.446053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:12.447658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:12.478766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc60bf7-215f-4f16-b0a1-8f82efbf8420 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:12.545631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:12.561198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:12.561898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.564251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.564555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] PATH_INFO: `/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.568514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c5d404194048a795b6e8b3ddf2f0fb)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:12.570658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:12.638666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c5d404194048a795b6e8b3ddf2f0fb', 'request_id': 'req-4b3ccd94-7725-4822-b415-85f6cecc39c8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.654674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c5d404194048a795b6e8b3ddf2f0fb', 'request_id': 'req-4b3ccd94-7725-4822-b415-85f6cecc39c8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.668393 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c5d404194048a795b6e8b3ddf2f0fb', 'request_id': 'req-4b3ccd94-7725-4822-b415-85f6cecc39c8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.683707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] The token cache is being invalidate because project e7c5d404194048a795b6e8b3ddf2f0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:12.684043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.685707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4b3ccd94-7725-4822-b415-85f6cecc39c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:12.689423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1012/2036] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:12 2022] DELETE /identity/v3/projects/e7c5d404194048a795b6e8b3ddf2f0fb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:12.696290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.697822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.698279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.726002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:12.727277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:12.807684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1025/2037] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:12.817508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.817823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.818175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.841392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:12.842754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:12.851266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1013/2038] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:12.855347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:12.865175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c1869-6fb4-465e-ae08-2e354eb762e0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:12.869838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:12.870465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.870740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.871012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.871666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:12.872083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:12.873820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:12.886855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1026/2039] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:12.891204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:12.900709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4fc73-5c5d-4781-8e93-a419fe1f55a5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:12.905452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:12.906092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:12.906372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:12.906663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:12.907279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:12.909227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:12.941139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1014/2040] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:12 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:12.945584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:12.956340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42920fb9-35d5-469c-8bbc-257e5ff15bf8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.010483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:13.017170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.017869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.018094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.018374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.019029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.021487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.048755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1027/2041] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:12 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:13.052989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.063891 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657dbeb-f18c-4545-8bed-7b8910936f06 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.069443 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.070100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.070378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.070653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.071305 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.072917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.084419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1015/2042] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:13.088417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.104411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7d2f8-b22c-4518-a8b4-126826970177 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.109821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.110436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.110717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.111126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.111908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.113613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f0218-2d36-4805-8528-7d5a5d308480 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.129476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1028/2043] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:13.130792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.145421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1950c6a-840f-437d-b453-bdb0c13586e1 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.150148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.150817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.151108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.151366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] PATH_INFO: `/v3/projects/c4df92b07c664942ad06ef61767f6809/users/7ad1c5e0ec874737ae28c7464f712c34/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.166628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4df92b07c664942ad06ef61767f6809, user_id=7ad1c5e0ec874737ae28c7464f712c34, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.169308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.186485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1016/2044] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:13 2022] PUT /identity/v3/projects/c4df92b07c664942ad06ef61767f6809/users/7ad1c5e0ec874737ae28c7464f712c34/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:13.197744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.198116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.198444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.225892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] MFA Rules not processed for user `7ad1c5e0ec874737ae28c7464f712c34`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:13.227100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:13.298254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1029/2045] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:13.303086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.372292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1304a-64e2-4c7d-9104-a98370a59973 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.379972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.380588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.380890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.381153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.382060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.384091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.415586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1017/2046] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:13 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:13.419758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.431716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec60af-d0e0-4917-8e53-d9464f63cd60 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.498185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.498811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.499060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.499319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.499937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.502004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.526451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1030/2047] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:13 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:13.530635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.542247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae75f-d49d-4f8d-946c-1432305e2fe5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.548338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.548948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.549204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.549474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.550180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.551836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.554391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:13.564157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1018/2048] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:13.568855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.579789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a2444-28f4-4633-83b8-ead11049e1b6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.584405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.585056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.585335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.585618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] PATH_INFO: `/v3/projects/0af1a4607cc143778a2196a28ada8025/users/cc4bd4b9dace4885abe466eed7f10468/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.601419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af1a4607cc143778a2196a28ada8025, user_id=cc4bd4b9dace4885abe466eed7f10468, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.604148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.619515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1031/2049] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:13 2022] PUT /identity/v3/projects/0af1a4607cc143778a2196a28ada8025/users/cc4bd4b9dace4885abe466eed7f10468/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:13.633608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.634944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad889d3c-bb3d-46cf-a098-6ec78ee3acd0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.699231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.699231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.699231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.699231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.700067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.701449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ea756-f237-445a-a40c-dd58fc9be6ad admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.712255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1019/2050] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:13.716522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.730051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82d5-dd34-42b8-b2e2-45b1e8b5e8b6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.736006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.736695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.736973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.737241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] PATH_INFO: `/v3/projects/0af1a4607cc143778a2196a28ada8025/users/cc4bd4b9dace4885abe466eed7f10468/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.739466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af1a4607cc143778a2196a28ada8025, user_id=cc4bd4b9dace4885abe466eed7f10468, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.743687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.766940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1032/2051] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:13 2022] PUT /identity/v3/projects/0af1a4607cc143778a2196a28ada8025/users/cc4bd4b9dace4885abe466eed7f10468/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:13.770444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.770711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.771002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.800625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] MFA Rules not processed for user `cc4bd4b9dace4885abe466eed7f10468`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:13.801935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:13.879243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1020/2052] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:13.891673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:13.894833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:13.916924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:13.956172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178b147-bf2d-4ea9-9ca7-85521d447aca admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:13.959934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:13.960534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:13.960804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:13.961066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:13.971345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:13.972859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ea4d0-73fb-4b89-821f-90927f6feab9 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:13.983114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1033/2053] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:14.030723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:14.032218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:14.046429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f7f9e-d6b5-4151-930e-d13937874772 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:14.051119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:14.051753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:14.052034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:14.052302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:14.110426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:14.111916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bb0b1-9d15-450f-85ad-f0513e3e3b7b service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:14.120993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1021/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:17.418237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:17.418778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:17.418980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:17.421290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1034/2055] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:04:17.441968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443103-9429-4f4a-91aa-e89f716bf3eb None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:17.442503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443103-9429-4f4a-91aa-e89f716bf3eb None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:17.442697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443103-9429-4f4a-91aa-e89f716bf3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:17.473294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1f443103-9429-4f4a-91aa-e89f716bf3eb None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:17.474604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1f443103-9429-4f4a-91aa-e89f716bf3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:17.488860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:17.545184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1022/2056] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:04:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:17.566724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:17.568669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:17.593980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a20f8-3956-4527-8941-5f7dae4d87b4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:17.599881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:17.600848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:17.602131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:17.602436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:17.612783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:17.614587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53190bf7-a802-4948-9371-1501547b6fc3 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:17.626384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1035/2057] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:18.242179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67089c-179d-45da-9cf8-8922e0ef7f6a None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.242543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67089c-179d-45da-9cf8-8922e0ef7f6a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.242823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67089c-179d-45da-9cf8-8922e0ef7f6a None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.244548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1023/2058] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:04:18.274871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.275207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.275480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.302702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] MFA Rules not processed for user `7abc4e612c0241ac8efb2789a5741250`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:18.304505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.316376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1036/2059] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:04:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:18.331643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67089c-179d-45da-9cf8-8922e0ef7f6a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.339161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.343011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.359105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67089c-179d-45da-9cf8-8922e0ef7f6a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:18.364583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:18.365395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.365724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.365996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.375275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:18.377107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:18.391875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1024/2060] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:18.431022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9ab42-ec25-45f0-ac8e-46e834a21958 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:18.440902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:18.441616 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.441990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.442470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] PATH_INFO: `/v3/users/050326a57dba45d8bc55dabac8d8b039` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.443636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050326a57dba45d8bc55dabac8d8b039)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:18.445724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:18.499227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050326a57dba45d8bc55dabac8d8b039'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.506687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050326a57dba45d8bc55dabac8d8b039'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.521435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050326a57dba45d8bc55dabac8d8b039'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.530981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050326a57dba45d8bc55dabac8d8b039'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.532153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050326a57dba45d8bc55dabac8d8b039'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.538749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1037/2061] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:18 2022] DELETE /identity/v3/users/050326a57dba45d8bc55dabac8d8b039 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:18.693530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.709355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bfb44-7f96-40a8-a235-84274f77fd1a service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:18.731815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.733498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.768384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:18.798175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:18.798820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.799101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.799374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] PATH_INFO: `/v3/users/7ad1c5e0ec874737ae28c7464f712c34` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.800520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad1c5e0ec874737ae28c7464f712c34)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:18.802502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:18.819527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ca20e-ed84-4a6a-a975-fd6bc1e97258 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:18.824123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:18.824753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.825028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.825300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] PATH_INFO: `/v3/projects/1e1a085fa5154c658a5b69e9dc85ce37` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.826465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1a085fa5154c658a5b69e9dc85ce37)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:18.828703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:18.850845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad1c5e0ec874737ae28c7464f712c34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.851524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad1c5e0ec874737ae28c7464f712c34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.852132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad1c5e0ec874737ae28c7464f712c34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.866846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad1c5e0ec874737ae28c7464f712c34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.874152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a16d8243-9896-4d41-bac2-341c0f9d101f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad1c5e0ec874737ae28c7464f712c34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.885380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1a085fa5154c658a5b69e9dc85ce37', 'request_id': 'req-79cf2564-c68d-469b-94e2-614aeac98dc1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.888484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1025/2062] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:18 2022] DELETE /identity/v3/users/7ad1c5e0ec874737ae28c7464f712c34 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:18.891395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1a085fa5154c658a5b69e9dc85ce37', 'request_id': 'req-79cf2564-c68d-469b-94e2-614aeac98dc1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.892817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:18.893196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:18.893473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:18.897975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1a085fa5154c658a5b69e9dc85ce37', 'request_id': 'req-79cf2564-c68d-469b-94e2-614aeac98dc1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.911620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] The token cache is being invalidate because project 1e1a085fa5154c658a5b69e9dc85ce37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:18.911923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.912543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:18.913243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:18.914514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.914874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1038/2063] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:18 2022] DELETE /identity/v3/projects/1e1a085fa5154c658a5b69e9dc85ce37 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:04:18.919073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.920376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.928602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:18.984246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1026/2064] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:18.992022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:18.993737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:18.996562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2564-c68d-469b-94e2-614aeac98dc1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.002818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.003475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.003753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.004024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] PATH_INFO: `/v3/users/e43b139327834956b755a04b454f2df6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.005254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43b139327834956b755a04b454f2df6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.007545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.056230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03ab6b-fa64-4b5e-9298-eb6d16a661ce None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.059469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43b139327834956b755a04b454f2df6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.059903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.060520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.060799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.061086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.063924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:19.067015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43b139327834956b755a04b454f2df6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.070670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.072383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.081102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43b139327834956b755a04b454f2df6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.088572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1027/2065] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:04:19.095014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43b139327834956b755a04b454f2df6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.095658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43b139327834956b755a04b454f2df6'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.102088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1039/2066] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:18 2022] DELETE /identity/v3/users/e43b139327834956b755a04b454f2df6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:19.326864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.339285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6f8da-8614-4d6b-95c4-487146065bbd service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.417146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.417819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.418099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.418395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] PATH_INFO: `/v3/projects/27fdf600d1d548e0a51256e12b8b932e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.421745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fdf600d1d548e0a51256e12b8b932e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.421745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.423340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.425044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:19.439361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4919-bf4a-4321-b4d1-e943f84e4bac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.444709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.445355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.445790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.446069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] PATH_INFO: `/v3/projects/c4df92b07c664942ad06ef61767f6809` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.447250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4df92b07c664942ad06ef61767f6809)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.449245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.496991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fdf600d1d548e0a51256e12b8b932e', 'request_id': 'req-f276aec5-d619-46b7-805c-4a6109f50f28'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.502769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fdf600d1d548e0a51256e12b8b932e', 'request_id': 'req-f276aec5-d619-46b7-805c-4a6109f50f28'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.511691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fdf600d1d548e0a51256e12b8b932e', 'request_id': 'req-f276aec5-d619-46b7-805c-4a6109f50f28'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.513037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:19.526110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4df92b07c664942ad06ef61767f6809', 'request_id': 'req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.527816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] The token cache is being invalidate because project 27fdf600d1d548e0a51256e12b8b932e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:19.528581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.529244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f276aec5-d619-46b7-805c-4a6109f50f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.530994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1028/2067] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:19 2022] DELETE /identity/v3/projects/27fdf600d1d548e0a51256e12b8b932e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:19.532545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4df92b07c664942ad06ef61767f6809', 'request_id': 'req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.535622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db08be-6f6e-4c92-9c94-0c116f60e196 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.536019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db08be-6f6e-4c92-9c94-0c116f60e196 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.536336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db08be-6f6e-4c92-9c94-0c116f60e196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.543794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4df92b07c664942ad06ef61767f6809', 'request_id': 'req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.560714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] The token cache is being invalidate because project c4df92b07c664942ad06ef61767f6809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:19.561182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.561977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.563411 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-19db08be-6f6e-4c92-9c94-0c116f60e196 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:19.564139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1040/2068] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:19 2022] DELETE /identity/v3/projects/c4df92b07c664942ad06ef61767f6809 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:19.564665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-19db08be-6f6e-4c92-9c94-0c116f60e196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:19.569604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.569926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:19.643646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1029/2069] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:19.646989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1dbd0-eeaa-4f31-8817-8bab9ff90a31 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.652113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911471a-341b-425b-a981-18e448000bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.652854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911471a-341b-425b-a981-18e448000bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.653191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.653308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911471a-341b-425b-a981-18e448000bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.653483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.653781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.653905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911471a-341b-425b-a981-18e448000bac admin admin] PATH_INFO: `/v3/users/cc4bd4b9dace4885abe466eed7f10468` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.655133 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911471a-341b-425b-a981-18e448000bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4bd4b9dace4885abe466eed7f10468)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.657367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911471a-341b-425b-a981-18e448000bac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.680484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:19.681934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:19.692147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1030/2070] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:19.696793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.706588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3df024-e136-4929-9b7d-1315748ed809 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.712657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.713298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.713623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.713893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.714604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:19.714908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.716493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.717695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4bd4b9dace4885abe466eed7f10468'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.723906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:19.726253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4bd4b9dace4885abe466eed7f10468'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.732896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1031/2071] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:04:19.734962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.742692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4bd4b9dace4885abe466eed7f10468'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.747929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fce52b-15f2-4072-acb1-ceeced2c5c17 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.751748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4bd4b9dace4885abe466eed7f10468'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.752522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4bd4b9dace4885abe466eed7f10468'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:19.753989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.754644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.754905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.755178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.755810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.757798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.759005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1041/2072] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:19 2022] DELETE /identity/v3/users/cc4bd4b9dace4885abe466eed7f10468 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:19.792333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1032/2073] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:19 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:19.793456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:19.794951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:19.885235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911471a-341b-425b-a981-18e448000bac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:19.962519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:19.963137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:19.963411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:19.963678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:19.964348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:19.966550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:19.994522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1042/2074] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:19 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:04:19.996579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.010244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76816771-1475-4568-9230-b2a84ead5516 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.015081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.015707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.015987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.016261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.016942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.018810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.034708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.040425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1033/2075] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:20.048023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b146d0-d382-41ea-b000-309360ac8a88 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.056227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.057407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.057407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.057720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.061831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.065976 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.075792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1043/2076] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:20.080449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.092476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c95087-865b-4ded-a050-f726def738f3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.094004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.096974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.102275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.103295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.104119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.104572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] PATH_INFO: `/v3/projects/04042df62e7941d7a6d9d1cf493d9602/users/1e6508646820461591015314de163c2f/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.125575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce54d3-578a-4183-94f3-306f144125dc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.125575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.125575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.125575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.126055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] PATH_INFO: `/v3/projects/0af1a4607cc143778a2196a28ada8025` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.126055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af1a4607cc143778a2196a28ada8025)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.126225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04042df62e7941d7a6d9d1cf493d9602, user_id=1e6508646820461591015314de163c2f, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.126316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.126399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309b709a-62c2-474d-894b-7f15a7120678 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.148836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1034/2077] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:20 2022] PUT /identity/v3/projects/04042df62e7941d7a6d9d1cf493d9602/users/1e6508646820461591015314de163c2f/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:20.153939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.154230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.154510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.174076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:20.175181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.180594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af1a4607cc143778a2196a28ada8025', 'request_id': 'req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:20.182587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:20.191544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af1a4607cc143778a2196a28ada8025', 'request_id': 'req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:20.199727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af1a4607cc143778a2196a28ada8025', 'request_id': 'req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:20.217293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] The token cache is being invalidate because project 0af1a4607cc143778a2196a28ada8025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:20.217700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:20.219061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-018500aa-e1c4-4da5-ac69-a949f0a7fbf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:20.225313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1044/2078] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:20 2022] DELETE /identity/v3/projects/0af1a4607cc143778a2196a28ada8025 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:20.227517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9f45-ddc6-4f05-9a41-f4cfb86d17a2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.227798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9f45-ddc6-4f05-9a41-f4cfb86d17a2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.228067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d9f45-ddc6-4f05-9a41-f4cfb86d17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.270444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1035/2079] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:20.273129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fa4d9f45-ddc6-4f05-9a41-f4cfb86d17a2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:20.274664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fa4d9f45-ddc6-4f05-9a41-f4cfb86d17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.280048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.282636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.284927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1045/2080] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:20.293790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.295388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.295772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.324250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:20.329208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:20.331026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.341753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1046/2081] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:20.346024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.357941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd40f8-fec4-421a-9f8e-8f797cfac35c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.370336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba8f53-dbe7-4eb7-9b16-b98dd3bc9948 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.374264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.374436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.375031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.375387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.375496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.375781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.375971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.376673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.378444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:20.379532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.385223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.394173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.396596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.406950 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1047/2082] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:04:20.410356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1036/2083] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:20.411565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.428057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64a6cd-04a5-4b5c-8111-dd91c39c878a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.433270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.433913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.434187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.434498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.435065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.437020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.472447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1048/2084] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:20.476821 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.486719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782539d1-e727-42f3-9b57-5385b50841c0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.555983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.556745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.557021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.557291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.557955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.559883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.589303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1037/2085] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:20.590371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.607778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfcc01-0840-40ac-bda6-94ca4d7359a1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.613452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.614560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.615591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.616078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.616669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.619103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.630946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1049/2086] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:20.635580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.648392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6528a-0988-4593-bbe7-657881f5c399 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.659113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.659747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.660026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.660301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.661018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.662682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821c76a-f054-4f33-a13f-65326705dcc9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.674504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1038/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:20.678859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.692338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a47511-4b1e-479b-a053-f551244de9d8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.698189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:20.698936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.699270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.699718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] PATH_INFO: `/v3/projects/5e7c5b9cd3994280abdabc9725252041/users/c88c84f6c7904230940cc6a57bdaad65/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.716563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7c5b9cd3994280abdabc9725252041, user_id=c88c84f6c7904230940cc6a57bdaad65, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:20.719601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:20.739027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1050/2088] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:20 2022] PUT /identity/v3/projects/5e7c5b9cd3994280abdabc9725252041/users/c88c84f6c7904230940cc6a57bdaad65/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:20.750212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:20.750693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:20.750693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:20.800117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] MFA Rules not processed for user `c88c84f6c7904230940cc6a57bdaad65`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:20.801529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:20.908264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1039/2089] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:20.913029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:20.991881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81933a8-babc-4852-94e8-87877bba5411 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:20.999494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.000064 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.000338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.000601 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.001895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.005088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.039277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1051/2090] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:20 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:21.043728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.056759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc70708-e6a1-4de3-9a2d-8d4dcce15547 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.094249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:21.133609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.134260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.135257 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.135511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.136269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.138475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06580cf-8aae-4348-ba90-e7d77b1ad1f7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.170820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1040/2091] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:21 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:21.174599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.190050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a4dc5-4b92-481b-88b7-c574501411cc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.196418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.197036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.197468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.197773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.198574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.200188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.204815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.205172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.205665 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.216859 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1052/2092] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:21.218055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.229737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442c638-6736-4cf8-a091-4448f7fa3e0c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.239359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.240029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.240302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.240568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] PATH_INFO: `/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe/users/b111de2d3fe24bccb1ecb7d35fad0b6e/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.247960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] MFA Rules not processed for user `1e6508646820461591015314de163c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:21.249646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:21.279218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578dc9a1b16d44b0bd714ec684e18bbe, user_id=b111de2d3fe24bccb1ecb7d35fad0b6e, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.282148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.322790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1053/2093] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:21 2022] PUT /identity/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe/users/b111de2d3fe24bccb1ecb7d35fad0b6e/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:21.323944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.340540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:21.348662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1041/2094] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:21.360080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.361479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:21.447852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba504d9-8218-4eec-8416-1a5cbf8a0b74 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.458812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153247cb-08c9-4e8b-b49c-834fd7da9512 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.462205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.462849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.463122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.463385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.464065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.465795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.476623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1054/2095] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:21.480834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.492078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc28665-6323-4cfc-872f-6bbc711496c7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.499482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.500768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.501664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.502121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] PATH_INFO: `/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe/users/b111de2d3fe24bccb1ecb7d35fad0b6e/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.505112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578dc9a1b16d44b0bd714ec684e18bbe, user_id=b111de2d3fe24bccb1ecb7d35fad0b6e, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.509059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fdf444-8fa5-46ae-826c-d25b94c807b0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.524152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1055/2096] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:21 2022] PUT /identity/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe/users/b111de2d3fe24bccb1ecb7d35fad0b6e/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:04:21.527796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.528424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.528771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.528957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.535384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99dc55-f045-430c-a403-7137567089dc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.535787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99dc55-f045-430c-a403-7137567089dc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.536129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99dc55-f045-430c-a403-7137567089dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.558357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:21.598276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0c99dc55-f045-430c-a403-7137567089dc None None] MFA Rules not processed for user `b111de2d3fe24bccb1ecb7d35fad0b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:21.601450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c99dc55-f045-430c-a403-7137567089dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:21.646213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.648390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.673614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1042/2097] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:21.698191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1056/2098] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:21.711542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.713020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:21.737921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad0356-98de-4c1a-9323-b5198c25830c service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:21.750209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d211c-d25f-4e69-a869-8728132eecca None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.750459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d211c-d25f-4e69-a869-8728132eecca None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.750797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d211c-d25f-4e69-a869-8728132eecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.771339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-955d211c-d25f-4e69-a869-8728132eecca None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:21.772560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-955d211c-d25f-4e69-a869-8728132eecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:21.807182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:21.807856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:21.808294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:21.808581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:21.826636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:21.828767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:21.841379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1043/2099] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:21.844641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1057/2100] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:04:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:21.918437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:21.919923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:22.002801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d211c-d25f-4e69-a869-8728132eecca None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:22.022112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d211c-d25f-4e69-a869-8728132eecca None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:22.027169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:22.027882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:22.028303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:22.028577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:22.050659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f656-9a69-4e3c-93df-e8c8249ee014 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:22.061501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:22.063145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:22.063783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:22.064062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:22.064341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:22.078253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:22.080439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b1768-9e89-4f1e-a81f-c07902b0c511 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:22.093224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1044/2101] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:04:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:04:22.123417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:22.124166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:22.136109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1058/2102] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:22.661619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166586-a5a6-4983-af7e-1ab3bfc7c10f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:22.662040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166586-a5a6-4983-af7e-1ab3bfc7c10f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:22.662421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee166586-a5a6-4983-af7e-1ab3bfc7c10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:22.687355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ee166586-a5a6-4983-af7e-1ab3bfc7c10f None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:22.688554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ee166586-a5a6-4983-af7e-1ab3bfc7c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:22.697164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1045/2103] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:04:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:22.710012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:22.722519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0b614-0e38-4014-bceb-907e75c223cc service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:22.728381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:22.728984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:22.729261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:22.729576 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:22.738008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:22.739649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:22.749213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1059/2104] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:04:22.991461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19845-8e6d-45a0-8cf6-c600c761c73d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:22.991949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19845-8e6d-45a0-8cf6-c600c761c73d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:22.992148 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19845-8e6d-45a0-8cf6-c600c761c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:23.011998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-13f19845-8e6d-45a0-8cf6-c600c761c73d None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:23.013227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13f19845-8e6d-45a0-8cf6-c600c761c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:23.021911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1046/2105] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:04:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:23.034307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:23.045253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa817e2-914f-45ba-9f46-d27d02ed122a service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:23.051132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:23.051755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:23.052030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:23.052298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:23.060570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:23.062326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030ab8e-4e90-4f1f-8f7e-070eb87654b2 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:23.070762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1060/2106] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:04:23.162004 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:23.162285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:23.162777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:23.182396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] MFA Rules not processed for user `1e6508646820461591015314de163c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:23.183609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:23.223034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:23.246439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1047/2107] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:23.467236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:23.467516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:23.467788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:23.487139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:23.488388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:23.496141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1061/2108] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:23.509249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:23.521758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b150fb5-5361-43b6-add6-32655ed29a44 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:23.527075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:23.527632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:23.527912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:23.528323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:23.548179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:23.549919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:23.559157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1048/2109] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:04:25.836943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:25.838566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:25.887924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:25.933436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f11d2a-4b73-4c42-84f5-912bceb8720c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:25.938713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:25.939360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:25.939635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:25.939903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] PATH_INFO: `/v3/users/7389adc9e92743ab921b375a2cbe117e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:25.941362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7389adc9e92743ab921b375a2cbe117e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:25.943599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:25.969267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae29a6f-57d7-4913-a487-d63f56fb7b8f service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:25.975330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:25.976408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:25.976726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:25.977108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] PATH_INFO: `/v3/users/c88c84f6c7904230940cc6a57bdaad65` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:25.978664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88c84f6c7904230940cc6a57bdaad65)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:25.980694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:26.028720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7389adc9e92743ab921b375a2cbe117e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.039307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7389adc9e92743ab921b375a2cbe117e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.048486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88c84f6c7904230940cc6a57bdaad65'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.054504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88c84f6c7904230940cc6a57bdaad65'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.055394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88c84f6c7904230940cc6a57bdaad65'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.060047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7389adc9e92743ab921b375a2cbe117e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.070057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88c84f6c7904230940cc6a57bdaad65'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.076349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7389adc9e92743ab921b375a2cbe117e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.078032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-21a1880c-4e61-4b2c-852e-390fb95c574b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7389adc9e92743ab921b375a2cbe117e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.079798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-c4233f33-5c33-4119-b12e-328561082795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88c84f6c7904230940cc6a57bdaad65'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.089086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1062/2110] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:25 2022] DELETE /identity/v3/users/7389adc9e92743ab921b375a2cbe117e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:26.097041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:26.097041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:26.097041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:26.102341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1049/2111] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:25 2022] DELETE /identity/v3/users/c88c84f6c7904230940cc6a57bdaad65 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:26.106551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:26.106915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:26.108017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:26.108319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:26.132945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:26.134215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.134940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:26.136457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.194828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:26.218668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1063/2112] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:26.222879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1050/2113] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:26.229952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:26.231445 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.232860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:26.234474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.298204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf889c6-e4c3-4ec0-9250-b964aead2855 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:26.306182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:26.306182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:26.306182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:26.306182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:26.314261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:26.315702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:26.327963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1051/2114] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:26.339852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d3dda-a530-4286-94cc-3147352d8331 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:26.344381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:26.345613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:26.345613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:26.345613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:26.357715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:26.359169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:26.370684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1064/2115] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:26.811577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:26.813565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.832446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91129d3d-210d-4b00-8219-ee8aa9f5430b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:26.840669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:26.841686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:26.842430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:26.843373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] PATH_INFO: `/v3/projects/997dd7a0c62947e09c6cab0cddcc75d5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:26.845056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997dd7a0c62947e09c6cab0cddcc75d5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:26.847500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:26.939790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:26.940387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997dd7a0c62947e09c6cab0cddcc75d5', 'request_id': 'req-7c70a016-5238-4276-bc25-c79b567d8bb4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.941571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:26.960005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997dd7a0c62947e09c6cab0cddcc75d5', 'request_id': 'req-7c70a016-5238-4276-bc25-c79b567d8bb4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.971849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95414a-3bb2-4d66-99f8-d3e6efff069f service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:26.974122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997dd7a0c62947e09c6cab0cddcc75d5', 'request_id': 'req-7c70a016-5238-4276-bc25-c79b567d8bb4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.989789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] The token cache is being invalidate because project 997dd7a0c62947e09c6cab0cddcc75d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:26.990465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.991591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:26.993338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1052/2116] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:26 2022] DELETE /identity/v3/projects/997dd7a0c62947e09c6cab0cddcc75d5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:27.041769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:27.042436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:27.042713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:27.042982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] PATH_INFO: `/v3/projects/5e7c5b9cd3994280abdabc9725252041` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:27.044499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7c5b9cd3994280abdabc9725252041)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:27.046243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:27.122934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7c5b9cd3994280abdabc9725252041', 'request_id': 'req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.124845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:27.130877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7c5b9cd3994280abdabc9725252041', 'request_id': 'req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.150389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7c5b9cd3994280abdabc9725252041', 'request_id': 'req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.178701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] The token cache is being invalidate because project 5e7c5b9cd3994280abdabc9725252041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:27.179385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.180958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.184106 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1065/2117] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:26 2022] DELETE /identity/v3/projects/5e7c5b9cd3994280abdabc9725252041 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:04:27.188817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:27.193682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:27.208250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70a016-5238-4276-bc25-c79b567d8bb4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:27.267852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:27.268503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:27.268854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:27.269035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] PATH_INFO: `/v3/users/b111de2d3fe24bccb1ecb7d35fad0b6e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:27.270275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b111de2d3fe24bccb1ecb7d35fad0b6e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:27.272202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:27.292040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:27.351951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b111de2d3fe24bccb1ecb7d35fad0b6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.352624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b111de2d3fe24bccb1ecb7d35fad0b6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.353336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b111de2d3fe24bccb1ecb7d35fad0b6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.365854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b111de2d3fe24bccb1ecb7d35fad0b6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.372498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a1bea20d-f95b-4caa-b732-1b9bfb68d563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b111de2d3fe24bccb1ecb7d35fad0b6e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.387315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1053/2118] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:27 2022] DELETE /identity/v3/users/b111de2d3fe24bccb1ecb7d35fad0b6e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:27.701968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:27.708986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:27.725400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113f3d3-3afe-49fe-a1ea-e4f4b7163fac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:27.810610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:27.811246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:27.811511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:27.811773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] PATH_INFO: `/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:27.813003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] RBAC: Authorizing `identity:delete_project(project_id=578dc9a1b16d44b0bd714ec684e18bbe)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:27.814885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:27.868353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578dc9a1b16d44b0bd714ec684e18bbe', 'request_id': 'req-013c18b2-16b1-45ec-9686-b12d285106af'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.874646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578dc9a1b16d44b0bd714ec684e18bbe', 'request_id': 'req-013c18b2-16b1-45ec-9686-b12d285106af'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.881958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578dc9a1b16d44b0bd714ec684e18bbe', 'request_id': 'req-013c18b2-16b1-45ec-9686-b12d285106af'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.896383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] The token cache is being invalidate because project 578dc9a1b16d44b0bd714ec684e18bbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:27.896689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.897292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-013c18b2-16b1-45ec-9686-b12d285106af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:27.902272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1066/2119] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:27 2022] DELETE /identity/v3/projects/578dc9a1b16d44b0bd714ec684e18bbe => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:27.903773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:27.904065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:27.904336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:27.924174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:27.925397 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:27.990185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1054/2120] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:27.998364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:27.998830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:27.998992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.019650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:28.020874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:28.029060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1067/2121] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:28.034047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.044813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5c371-9f02-41dd-8e73-bf3fa5cc58a0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.050392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.051202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.051519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.051794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.052456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:28.052762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.054398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.066828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1055/2122] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:28.071151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.080546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91178b-edad-4fce-9d20-6bcf0585deb0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.085347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.085981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.086251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.086513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.087144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.089054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.118404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1068/2123] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:28.123180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.139017 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf96f-c447-4c6e-8804-7846bf182346 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.200553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.201256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.201554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.201862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.202493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.204394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.228491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1056/2124] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:04:28.234057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.245031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82eecb8-9657-4b89-a65e-ed5a37def095 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.250400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.250998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.251266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.251527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.252167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.253797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.264909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1069/2125] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:28.269295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.280140 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438e62-9d12-4a24-982d-59a92dbfda96 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.285144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.285804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.286079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.286345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.286981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.288581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cff235-a4a0-4dd2-9097-e43e3f722f7a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.294370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:28.301130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1057/2126] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:28.305386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.316624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942afe-33c4-4b7a-ac42-fbe6ea4a6b91 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.321713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.322454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.322788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.323114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] PATH_INFO: `/v3/projects/dd26b60e650b47d5b1c03450ca84d79d/users/b8b592bf6d0c40308ece84fe5eefc6f1/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.338712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd26b60e650b47d5b1c03450ca84d79d, user_id=b8b592bf6d0c40308ece84fe5eefc6f1, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.341518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.355882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1070/2127] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:28 2022] PUT /identity/v3/projects/dd26b60e650b47d5b1c03450ca84d79d/users/b8b592bf6d0c40308ece84fe5eefc6f1/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:28.366463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.366745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.367015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.395701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] MFA Rules not processed for user `b8b592bf6d0c40308ece84fe5eefc6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:28.396921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:28.464568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1058/2128] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:28.469920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.476173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:28.538991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1bb7-7d24-499b-a345-967bfe2794e8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.543965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.544566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.544841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.545107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.545765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.548042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.578104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1071/2129] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:28.582268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.593639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bbcc8-6906-4d37-8dfa-6d61d23dd48d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.655735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.656367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.656643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.656916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.657565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.659504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.681612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1059/2130] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:28.685791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.696317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368280b3-783d-4f04-a192-c6020c8f5abe admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.701553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.702229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.702503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.702765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.703405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.705014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.716185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1072/2131] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:28.720301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.730497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca191c-fb5a-42e7-9cb7-4115f22a6863 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.735407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.736064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.736354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.736606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] PATH_INFO: `/v3/projects/add13734ae674a439f5308a99faede7e/users/fd39884b09f540aebf614ff80bed0902/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.755019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add13734ae674a439f5308a99faede7e, user_id=fd39884b09f540aebf614ff80bed0902, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.758519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.773882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1060/2132] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:28 2022] PUT /identity/v3/projects/add13734ae674a439f5308a99faede7e/users/fd39884b09f540aebf614ff80bed0902/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:28.777953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.788093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53f68-fa95-4343-9528-b0ca2a8e387f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.852474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.853384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.853855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.854262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.855283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.857596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74eab9-a8cd-40e7-b6ea-138aa498e2e9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.869800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1073/2133] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:28.874322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:28.885395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c1147-b6ee-4913-9a94-c92065ab89f7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:28.891945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:28.892295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.892890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.893269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] PATH_INFO: `/v3/projects/add13734ae674a439f5308a99faede7e/users/fd39884b09f540aebf614ff80bed0902/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.895811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=add13734ae674a439f5308a99faede7e, user_id=fd39884b09f540aebf614ff80bed0902, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:28.898521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:28.911083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1061/2134] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:28 2022] PUT /identity/v3/projects/add13734ae674a439f5308a99faede7e/users/fd39884b09f540aebf614ff80bed0902/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:28.922602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3a33-d293-4221-b423-84762ad408cb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:28.922885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3a33-d293-4221-b423-84762ad408cb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:28.923151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3a33-d293-4221-b423-84762ad408cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:28.952666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-761c3a33-d293-4221-b423-84762ad408cb None None] MFA Rules not processed for user `fd39884b09f540aebf614ff80bed0902`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:28.953957 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-761c3a33-d293-4221-b423-84762ad408cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:29.022861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:29.026790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1074/2135] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:29.038533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:29.040340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:29.093089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:29.106957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aa55c-2d66-416c-b674-d6b8de3fd8b8 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:29.111473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:29.112086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:29.112379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:29.112649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:29.173079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:29.174547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:29.183738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1062/2136] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:32.999963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c3a33-d293-4221-b423-84762ad408cb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.001469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-761c3a33-d293-4221-b423-84762ad408cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:33.068196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c3a33-d293-4221-b423-84762ad408cb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.072807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.076055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.076359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.076634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] PATH_INFO: `/v3/users/b095ab3c72b14c67adf61ab4189e4892` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.077866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b095ab3c72b14c67adf61ab4189e4892)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.079801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.139098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b095ab3c72b14c67adf61ab4189e4892'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.147831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b095ab3c72b14c67adf61ab4189e4892'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.168026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b095ab3c72b14c67adf61ab4189e4892'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.179220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b095ab3c72b14c67adf61ab4189e4892'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.181146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e92cf11c-b259-49e8-9e02-d16cac1d08aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b095ab3c72b14c67adf61ab4189e4892'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.188584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1075/2137] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:32 2022] DELETE /identity/v3/users/b095ab3c72b14c67adf61ab4189e4892 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:33.391441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.403495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1dd8d-b967-41db-9bda-a868257fad74 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.464214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.465054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.465341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.465641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] PATH_INFO: `/v3/projects/5433b5d7443545249b9329f1ce9e3e02` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.466787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5433b5d7443545249b9329f1ce9e3e02)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.468694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.516206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5433b5d7443545249b9329f1ce9e3e02', 'request_id': 'req-d244b4e8-8a1c-467b-896b-e9145a35c64e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.522488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5433b5d7443545249b9329f1ce9e3e02', 'request_id': 'req-d244b4e8-8a1c-467b-896b-e9145a35c64e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.532433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5433b5d7443545249b9329f1ce9e3e02', 'request_id': 'req-d244b4e8-8a1c-467b-896b-e9145a35c64e'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.554248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] The token cache is being invalidate because project 5433b5d7443545249b9329f1ce9e3e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:33.555076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.556221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d244b4e8-8a1c-467b-896b-e9145a35c64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:33.558898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1063/2138] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:33 2022] DELETE /identity/v3/projects/5433b5d7443545249b9329f1ce9e3e02 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:33.563869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.564394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.564884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.587019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:33.588283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:33.654719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1076/2139] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:33.663002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.663290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.663564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.685062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:33.686407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:33.695160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1064/2140] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:33.699652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.710891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f60732-9442-4b90-8f93-6e2ac6b4916b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.715559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.716155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.716425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.716780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.717566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:33.717868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.719460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.732019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1077/2141] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:04:33.736971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.751289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a701-9285-47ab-81f6-26643da5a737 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.757511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.758164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.758439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.758709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.759350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.761282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.785379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:33.796618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1065/2142] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:33 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:33.798175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.808255 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce66cf-b703-46e0-8d68-fb61d8994551 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.811863 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:33.868289 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.868898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.869173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.869441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.870081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.872040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.896999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1078/2143] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:33 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:33.898335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.908526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c9bc2-5d95-40f8-8e70-fa765e508ee5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.913125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.913757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.914034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.914282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.914951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.916538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.927217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1066/2144] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:33.931157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.941410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35be80-7454-4f78-a507-50397b40eaad admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.946486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.947076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.947348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.947612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.948248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:33.949891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbdedf-b1f7-4264-b783-98c65fca7d0b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:33.962424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1079/2145] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:33.966783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:33.976827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df4381-94b0-46f6-83d4-4e228b072d4d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:33.981952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:33.982589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:33.982879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:33.983158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] PATH_INFO: `/v3/projects/0e34f10f87da4be28fee390c918752ba/users/13ee33eb6c77478c9e12e4ffc8a192ca/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:33.998355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e34f10f87da4be28fee390c918752ba, user_id=13ee33eb6c77478c9e12e4ffc8a192ca, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:34.002604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:34.017037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1067/2146] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:33 2022] PUT /identity/v3/projects/0e34f10f87da4be28fee390c918752ba/users/13ee33eb6c77478c9e12e4ffc8a192ca/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:34.027968 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:34.028474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:34.028774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:34.058205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] MFA Rules not processed for user `13ee33eb6c77478c9e12e4ffc8a192ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:34.059407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:34.128238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1080/2147] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:34.139590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:34.141499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:34.200816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f38f3d-dc81-4bc5-b631-810169b70f9c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:34.204447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:34.205057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:34.205337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:34.205637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:34.214685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:34.216196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a197a-66fc-40e1-b3c5-dea383abe429 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:34.227333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1068/2148] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:36.740942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:36.751501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ef7bf-5383-4fbb-8f7e-fd3243cdc463 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:36.755747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:36.758204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:36.758570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:36.758840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:36.760787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:36.825347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:36.826776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:36.836409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1081/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:44.221793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:46.832044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:49.358810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907bf7c-1ea7-421f-8f64-92d297b424b0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:49.359343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907bf7c-1ea7-421f-8f64-92d297b424b0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:49.359343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907bf7c-1ea7-421f-8f64-92d297b424b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:49.378749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c907bf7c-1ea7-421f-8f64-92d297b424b0 None None] MFA Rules not processed for user `1e6508646820461591015314de163c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:49.379946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c907bf7c-1ea7-421f-8f64-92d297b424b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:49.437883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1069/2150] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:52.504331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:52.505860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:52.604454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b6fb3-856a-43da-83fc-868662391e50 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:52.610970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:52.611756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:52.612127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:52.612534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] PATH_INFO: `/v3/users/b8b592bf6d0c40308ece84fe5eefc6f1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:52.613704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b592bf6d0c40308ece84fe5eefc6f1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:52.615772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:52.670891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b592bf6d0c40308ece84fe5eefc6f1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:52.680651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b592bf6d0c40308ece84fe5eefc6f1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:52.697161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b592bf6d0c40308ece84fe5eefc6f1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:52.709155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b592bf6d0c40308ece84fe5eefc6f1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:52.710944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b592bf6d0c40308ece84fe5eefc6f1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:52.721337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1082/2151] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:52 2022] DELETE /identity/v3/users/b8b592bf6d0c40308ece84fe5eefc6f1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:52.726530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:52.727027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:52.727027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:52.762336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:52.774767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:52.854793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1070/2152] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:52.865681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:52.867165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:52.948391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284076-4036-4649-949e-48756cff61ad admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:52.952596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:52.953649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:52.954128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:52.954803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:52.964558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:52.966170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:52.976128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1083/2153] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:53.426169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:53.428247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:53.447655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424019e-4c21-4749-85a2-bd26de71ef00 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:53.454032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:53.454754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:53.455097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:53.455363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] PATH_INFO: `/v3/projects/dd26b60e650b47d5b1c03450ca84d79d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:53.456499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd26b60e650b47d5b1c03450ca84d79d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:53.458885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:53.520674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd26b60e650b47d5b1c03450ca84d79d', 'request_id': 'req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.527474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd26b60e650b47d5b1c03450ca84d79d', 'request_id': 'req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.537516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd26b60e650b47d5b1c03450ca84d79d', 'request_id': 'req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.551736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] The token cache is being invalidate because project dd26b60e650b47d5b1c03450ca84d79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:53.552023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.552681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.554430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1071/2154] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:53 2022] DELETE /identity/v3/projects/dd26b60e650b47d5b1c03450ca84d79d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:53.558506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:53.576462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc7584-a2de-4345-8241-47ae977d92d7 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:53.612105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:53.655304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:53.655936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:53.656206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:53.656470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] PATH_INFO: `/v3/users/fd39884b09f540aebf614ff80bed0902` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:53.657611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd39884b09f540aebf614ff80bed0902)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:53.659539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:53.711264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd39884b09f540aebf614ff80bed0902'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.718414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd39884b09f540aebf614ff80bed0902'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.730526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd39884b09f540aebf614ff80bed0902'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.748637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd39884b09f540aebf614ff80bed0902'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.749369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-73b5f3c4-ece3-46eb-9642-43ef185ffbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd39884b09f540aebf614ff80bed0902'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:53.755890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1084/2155] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:53 2022] DELETE /identity/v3/users/fd39884b09f540aebf614ff80bed0902 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:54.057937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.059546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.078629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846ff1a-ed0a-4e7a-b949-8a2a20912fdd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.107763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:54.141405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.142086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.142366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.142638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] PATH_INFO: `/v3/projects/add13734ae674a439f5308a99faede7e` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.143768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add13734ae674a439f5308a99faede7e)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.145713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.195832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add13734ae674a439f5308a99faede7e', 'request_id': 'req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.202027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add13734ae674a439f5308a99faede7e', 'request_id': 'req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.210904 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add13734ae674a439f5308a99faede7e', 'request_id': 'req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.223344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] The token cache is being invalidate because project add13734ae674a439f5308a99faede7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:54.223769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.224468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a276ce46-3b1f-4ef3-a2fa-d899b3b5ab68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.226178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1072/2156] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:54 2022] DELETE /identity/v3/projects/add13734ae674a439f5308a99faede7e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:54.232196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.232484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.232759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.261555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:54.262832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.336442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1085/2157] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:54.344693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.344980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.345256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.365348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:54.366669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.374670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1073/2158] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:54.378937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.388712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b6096-7b55-4215-8a1b-9d426880fcb6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.393362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.393988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.394263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.394525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.395171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:54.395469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.397079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.408831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1086/2159] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:54.413479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.423770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be29e9-5cc9-44a6-8ec7-13d33eaf2d75 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.429362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.430003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.430351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.430538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.431188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.433101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.464290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1074/2160] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:54.468752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.478750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d6090-77c3-4038-b549-898ba21ff7fc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.505472 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:54.539285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.539925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.540197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.540518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.541132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.543192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.568161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1087/2161] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:54.572523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.583038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11855ca6-19e5-4db3-ad05-0c3f38ee301e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.587809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.588413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.588693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.589292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.589638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.591236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.603247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1075/2162] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:54.606112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.616452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09a5d5-350b-4832-b8e5-844b90aaabef admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.617910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.619741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.621662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.622352 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.622711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.623049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.623768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.625468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.635854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009bb03-8347-4949-9cec-08eb29ee52c3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.638182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1088/2163] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:54.642416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.642814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.643533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.643731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.643991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] PATH_INFO: `/v3/users/13ee33eb6c77478c9e12e4ffc8a192ca` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.645181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ee33eb6c77478c9e12e4ffc8a192ca)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.647259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.653310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e1749-e0b0-4795-aba8-bab45d816361 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.660026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.660709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.660978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.661429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] PATH_INFO: `/v3/projects/ff864da3319b40fda1eefc641624b8a8/users/a09e946d053f49fdaa11da1b63260a08/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.679457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff864da3319b40fda1eefc641624b8a8, user_id=a09e946d053f49fdaa11da1b63260a08, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.683253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c994-20f7-415e-9ab7-fd1b807e7df7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.704586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ee33eb6c77478c9e12e4ffc8a192ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.705270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ee33eb6c77478c9e12e4ffc8a192ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.705617 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1089/2164] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:54 2022] PUT /identity/v3/projects/ff864da3319b40fda1eefc641624b8a8/users/a09e946d053f49fdaa11da1b63260a08/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:04:54.705947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ee33eb6c77478c9e12e4ffc8a192ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.714137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.714639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.715294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.720814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ee33eb6c77478c9e12e4ffc8a192ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.729202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c591818-0af5-4eca-9b36-4c90d24c68ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ee33eb6c77478c9e12e4ffc8a192ca'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:54.731522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:54.746863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1076/2165] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:54 2022] DELETE /identity/v3/users/13ee33eb6c77478c9e12e4ffc8a192ca => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:54.748801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] MFA Rules not processed for user `a09e946d053f49fdaa11da1b63260a08`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:54.751134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.752075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.752502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.752828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.783239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:54.784498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.828008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1090/2166] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:54.841911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.843308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.863147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1077/2167] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:54.870984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:54.873270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:54.901128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363c4cb-79b1-44fc-a583-8931dbc2c9a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.905454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.907947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.908225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.908491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.918215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.919679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.930501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1091/2168] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:04:54.939960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd40d5f-f565-456d-ac31-9106ca37cfea None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:54.943751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:54.944366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:54.944899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:54.944899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:54.954998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:54.956433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad0ca5-5b83-406d-a9b7-03393bafc3b7 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:54.967448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1078/2169] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:04:55.261989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.261989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:55.279325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfd3bc-aa99-41b5-8a25-189df63b90e2 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.284542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.285043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.285206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.285434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] PATH_INFO: `/v3/projects/0e34f10f87da4be28fee390c918752ba` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.286687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e34f10f87da4be28fee390c918752ba)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.288874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.302902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:55.341368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e34f10f87da4be28fee390c918752ba', 'request_id': 'req-e03686c4-1522-4337-9939-86d51cf283f3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:55.347128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e34f10f87da4be28fee390c918752ba', 'request_id': 'req-e03686c4-1522-4337-9939-86d51cf283f3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:55.361051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e34f10f87da4be28fee390c918752ba', 'request_id': 'req-e03686c4-1522-4337-9939-86d51cf283f3'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:55.367061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] The token cache is being invalidate because project 0e34f10f87da4be28fee390c918752ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:55.367371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:55.367969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e03686c4-1522-4337-9939-86d51cf283f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:55.369664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1092/2170] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:55 2022] DELETE /identity/v3/projects/0e34f10f87da4be28fee390c918752ba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:04:55.375682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.376178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.376295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.397384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:55.398607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:55.460085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1079/2171] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:55.469403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.469724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.469998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.490603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:55.491842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:55.500677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1093/2172] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:55.505160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.515830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482abdfd-0d9b-481e-a57b-3f22e0dfc5e5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.521789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.522421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.522725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.522993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.523655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:04:55.523958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.525708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.543063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1080/2173] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:04:55.545018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.557090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312ab91-7672-4e2b-b73d-04368c750fe6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.562983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.563592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.563869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.564135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.564768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.566795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.599393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1094/2174] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:55 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:55.603685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.614136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb37f5-4193-4329-a4fd-f6d692c00197 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.633577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:55.672322 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.673279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.673584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.673838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.674459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.676373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.698639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1081/2175] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:55 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:55.703017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.713287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b7bf3-940d-4690-936b-6cbadb9763aa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.718244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.718835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.719104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.719370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.720013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.721643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.732156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1095/2176] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:55.736233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.746409 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4bb6a-88fc-41a1-848a-dc087395b76f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.751613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.752202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.752493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.752745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.753385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.754998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6712581-e1d5-4e8a-b382-cc5193631061 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.766035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1082/2177] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:55.770337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.781183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adca5c-9963-4cb0-80d7-56e83ee1aa06 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:55.791716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:55.792876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.793156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.793425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] PATH_INFO: `/v3/projects/dd7ef76322934295bc52b1720bf4cb40/users/cc4eea4fa4254d50952aedee746a0174/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.810904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7ef76322934295bc52b1720bf4cb40, user_id=cc4eea4fa4254d50952aedee746a0174, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:55.813677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:55.833514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1096/2178] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:55 2022] PUT /identity/v3/projects/dd7ef76322934295bc52b1720bf4cb40/users/cc4eea4fa4254d50952aedee746a0174/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:55.835605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8d89-32eb-455e-8a11-e03928b21ddc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:55.835869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8d89-32eb-455e-8a11-e03928b21ddc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:55.836141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4d8d89-32eb-455e-8a11-e03928b21ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:55.856046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-2b4d8d89-32eb-455e-8a11-e03928b21ddc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:55.857341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-2b4d8d89-32eb-455e-8a11-e03928b21ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:55.917503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1083/2179] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:04:55.928538 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:55.930376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:55.996324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d668d6-5570-4a5d-b224-f6d931abab30 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:56.001376 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:56.002509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:56.002509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:56.003008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:56.014619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:56.016204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:56.027878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1097/2180] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:04:56.673831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:56.674386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:56.674386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:56.703167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] MFA Rules not processed for user `cc4eea4fa4254d50952aedee746a0174`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:56.704375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:56.774739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1084/2181] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:56 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:56.779295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:56.789409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fc25a-3604-48a0-a632-3bb07497e929 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:56.795188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:56.795793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:56.796066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:56.796329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:56.796963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:56.799315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:56.830970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1098/2182] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:04:56 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:04:56.832171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:56.844348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b119ed-2c0c-45b8-8392-d1929b5d28d5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:56.885131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:56.915581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:56.916108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:56.916381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:56.916650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:56.917278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:56.919231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:56.949521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1085/2183] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:04:56 2022] POST /identity/v3/users => generated 309 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:04:56.956372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:56.974058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1cd2a-e709-49c8-9748-adb4eaf663a3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:56.981566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:56.982204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:56.982470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:56.982737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:56.983391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:56.985014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:56.995947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1099/2184] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:57.000712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:57.011019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba12bf-2735-4a20-a6df-c0344eecf64b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:57.016400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:57.017056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:57.017330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:57.017629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] PATH_INFO: `/v3/projects/4ccb6e9e72c84c3786172f6d699f2959/users/2e0c74ab08f04d63bd33be3a9dde0101/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:57.033301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccb6e9e72c84c3786172f6d699f2959, user_id=2e0c74ab08f04d63bd33be3a9dde0101, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:57.035985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:57.051550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1086/2185] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:56 2022] PUT /identity/v3/projects/4ccb6e9e72c84c3786172f6d699f2959/users/2e0c74ab08f04d63bd33be3a9dde0101/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:57.055537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:57.062032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:57.065963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a8aa57-cd2f-4598-9e04-375e187d703a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:57.123606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:57.124217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:57.124489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:57.124754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:57.125395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:57.127026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5b58f-f137-44a0-a825-51c17f12dc47 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:57.137722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1100/2186] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:04:57.142021 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:57.152213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bcda9b-2b6a-4771-aedc-2faf45e9c32a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:57.156878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:57.157521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:57.157820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:57.158090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] PATH_INFO: `/v3/projects/4ccb6e9e72c84c3786172f6d699f2959/users/2e0c74ab08f04d63bd33be3a9dde0101/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:57.159981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccb6e9e72c84c3786172f6d699f2959, user_id=2e0c74ab08f04d63bd33be3a9dde0101, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:57.162653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:57.174200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1087/2187] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:04:57 2022] PUT /identity/v3/projects/4ccb6e9e72c84c3786172f6d699f2959/users/2e0c74ab08f04d63bd33be3a9dde0101/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:57.991014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:57.991578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:57.991578 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:58.023478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] MFA Rules not processed for user `2e0c74ab08f04d63bd33be3a9dde0101`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:04:58.024829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:58.095855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1101/2188] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:04:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:04:59.397018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:59.470095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8eb5-ed80-47b6-86c2-fc870e85f884 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:59.475224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:59.475930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:59.476204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:59.476480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] PATH_INFO: `/v3/users/cc4eea4fa4254d50952aedee746a0174` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:59.477676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4eea4fa4254d50952aedee746a0174)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:59.479594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:59.533934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4eea4fa4254d50952aedee746a0174'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.534623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4eea4fa4254d50952aedee746a0174'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.535245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4eea4fa4254d50952aedee746a0174'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.549282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4eea4fa4254d50952aedee746a0174'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.557438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4eea4fa4254d50952aedee746a0174'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.573564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1088/2189] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:59 2022] DELETE /identity/v3/users/cc4eea4fa4254d50952aedee746a0174 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:59.754655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:59.767127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed039d1-cd63-409a-8e6a-0a4f99060547 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:04:59.837211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:04:59.838517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:04:59.839291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:04:59.840062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] PATH_INFO: `/v3/projects/dd7ef76322934295bc52b1720bf4cb40` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:04:59.842061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7ef76322934295bc52b1720bf4cb40)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:04:59.844936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:04:59.882911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:59.902813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7ef76322934295bc52b1720bf4cb40', 'request_id': 'req-5c23fefd-0bc9-4b5f-9216-ccde187a7757'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.909312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7ef76322934295bc52b1720bf4cb40', 'request_id': 'req-5c23fefd-0bc9-4b5f-9216-ccde187a7757'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.916783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7ef76322934295bc52b1720bf4cb40', 'request_id': 'req-5c23fefd-0bc9-4b5f-9216-ccde187a7757'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.934222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] The token cache is being invalidate because project dd7ef76322934295bc52b1720bf4cb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:04:59.934642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.935790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:04:59.937703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1102/2190] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:04:59 2022] DELETE /identity/v3/projects/dd7ef76322934295bc52b1720bf4cb40 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:04:59.941417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:04:59.942865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:04:59.951060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:04:59.956810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e971-0256-431d-b9c9-d9bf42f2598f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.021375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.022058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.022342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.022756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] PATH_INFO: `/v3/users/2e0c74ab08f04d63bd33be3a9dde0101` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.023897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0c74ab08f04d63bd33be3a9dde0101)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.025935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.077039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0c74ab08f04d63bd33be3a9dde0101'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.077784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0c74ab08f04d63bd33be3a9dde0101'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.078495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0c74ab08f04d63bd33be3a9dde0101'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.090510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0c74ab08f04d63bd33be3a9dde0101'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.097685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3c0da0a6-977e-4895-bd57-e219a02c9ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0c74ab08f04d63bd33be3a9dde0101'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.111681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1089/2191] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:04:59 2022] DELETE /identity/v3/users/2e0c74ab08f04d63bd33be3a9dde0101 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:00.288340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.289765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:00.305248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23fefd-0bc9-4b5f-9216-ccde187a7757 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.363501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.364141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.364414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.364680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] PATH_INFO: `/v3/projects/4ccb6e9e72c84c3786172f6d699f2959` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.365825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccb6e9e72c84c3786172f6d699f2959)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.367737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.415157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccb6e9e72c84c3786172f6d699f2959', 'request_id': 'req-b520c8e2-26da-4b48-a8a4-cd805424b019'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.420810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccb6e9e72c84c3786172f6d699f2959', 'request_id': 'req-b520c8e2-26da-4b48-a8a4-cd805424b019'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.427506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccb6e9e72c84c3786172f6d699f2959', 'request_id': 'req-b520c8e2-26da-4b48-a8a4-cd805424b019'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.441988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] The token cache is being invalidate because project 4ccb6e9e72c84c3786172f6d699f2959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:00.442304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.442956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b520c8e2-26da-4b48-a8a4-cd805424b019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:00.444505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1103/2192] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:00 2022] DELETE /identity/v3/projects/4ccb6e9e72c84c3786172f6d699f2959 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:00.449687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780136a-9329-4bb4-a949-c0020317694c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.450163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780136a-9329-4bb4-a949-c0020317694c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.450283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780136a-9329-4bb4-a949-c0020317694c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.470406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5780136a-9329-4bb4-a949-c0020317694c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:00.471626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5780136a-9329-4bb4-a949-c0020317694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:00.540575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1090/2193] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:00.546165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.546165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.546165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.574385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:00.575614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:00.587590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1104/2194] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:00.588736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780136a-9329-4bb4-a949-c0020317694c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.599332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780136a-9329-4bb4-a949-c0020317694c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.604634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.605405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.605822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.606173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.607884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:00.608303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.610228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.627287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1091/2195] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:05:00.628413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.640200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39f508-2e47-48d5-9300-a29639144e85 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.647764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.648717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.649196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.649723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.650486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.652604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.686638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1105/2196] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:00 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:00.687839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.699738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88152c05-e9da-4bf4-9815-530248661383 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.704365 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:00.760174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.760795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.761081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.761351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.762028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.763971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.787438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1092/2197] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:00 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:00.791504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.802830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376a3e-4fb7-4eb7-9980-a498fbc8d19a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.807920 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.808512 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.808782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.809041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.809715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.811302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.821199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1106/2198] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:00.825413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.835543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f434af8-741f-429b-877a-eddf002a6e95 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.840611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.841193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.841445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.841770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.842406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.843981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bdf84-58b1-481c-a38f-58f9f8a746e5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.854936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1093/2199] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:00.859127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:00.869873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0fe6d-19fa-4b20-9e09-9979a8e59a94 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:00.875110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:00.876209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.876209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.876371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] PATH_INFO: `/v3/projects/54e4914c827b4e8b818ee8a0d5516077/users/bb6caf2d236543a2833125605c7d91b1/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.878734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:00.894074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54e4914c827b4e8b818ee8a0d5516077, user_id=bb6caf2d236543a2833125605c7d91b1, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:00.897229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:00.916602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1107/2200] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:00 2022] PUT /identity/v3/projects/54e4914c827b4e8b818ee8a0d5516077/users/bb6caf2d236543a2833125605c7d91b1/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:00.927330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:00.927612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:00.927878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:00.957743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] MFA Rules not processed for user `bb6caf2d236543a2833125605c7d91b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:00.959981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:01.029764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1094/2201] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:01.034530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.139609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d9872-f21d-44b7-924f-85b0634b62d4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.149946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.150694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.151038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.153159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.153959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.156182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.197194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1108/2202] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:01 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:01.202070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.221354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75470f4-7a41-4651-8ab5-c07c799b83ba None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.304745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.305366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.305678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.306020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.306631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.308555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.332037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1095/2203] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:01 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:01.336097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.347026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75f423-5de5-497f-b140-1b8fb11c4e33 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.352098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.352758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.353017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.353282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.353970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.355697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.367695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1109/2204] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:01.372550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.383302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d100a50-2093-4836-ba96-02702a4c44a9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.388381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.389031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.389316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.389615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] PATH_INFO: `/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7/users/69f0dcb8f1ce4cacadac9f71f59d3f4c/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.392093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:01.410241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c1bdd24bed427cb1e1b48d5049d0e7, user_id=69f0dcb8f1ce4cacadac9f71f59d3f4c, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.410668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.427029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1096/2205] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:01 2022] PUT /identity/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7/users/69f0dcb8f1ce4cacadac9f71f59d3f4c/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:01.431400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.442149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b501eb-7897-47aa-9b5a-3c47eae8c5d1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.502374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.503001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.503278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.503551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.504187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.505847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180e75-a22e-47d6-b384-29242db035f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.516967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1110/2206] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:01.521551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.532577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b4b14-7677-4c9c-bcb1-82b54c172c03 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.541691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.541691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.541691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.541691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] PATH_INFO: `/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7/users/69f0dcb8f1ce4cacadac9f71f59d3f4c/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.542068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c1bdd24bed427cb1e1b48d5049d0e7, user_id=69f0dcb8f1ce4cacadac9f71f59d3f4c, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.544865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.570128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.570415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.570691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.573349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1097/2207] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:01 2022] PUT /identity/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7/users/69f0dcb8f1ce4cacadac9f71f59d3f4c/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:01.605819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] MFA Rules not processed for user `69f0dcb8f1ce4cacadac9f71f59d3f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:01.607199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:01.687780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:01.773626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1111/2208] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:01.833243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:01.835023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:01.930685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a5188-6500-4a6f-9440-1642efbaa258 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:01.936046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:01.937317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:01.937697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:01.938069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:01.949475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:01.950982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630705ee-549a-4da9-a846-447f0eed2009 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:01.965578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1098/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:02.408448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:02.473278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a6fcf-075b-43e9-8136-347e3b3bdaed None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:02.478075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:02.478716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:02.478984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:02.479252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] PATH_INFO: `/v3/users/bb6caf2d236543a2833125605c7d91b1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:02.480396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6caf2d236543a2833125605c7d91b1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:02.482379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:02.538781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6caf2d236543a2833125605c7d91b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:02.546692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6caf2d236543a2833125605c7d91b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:02.563330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6caf2d236543a2833125605c7d91b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:02.572906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6caf2d236543a2833125605c7d91b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:02.573710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6caf2d236543a2833125605c7d91b1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:02.583071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1112/2210] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:02 2022] DELETE /identity/v3/users/bb6caf2d236543a2833125605c7d91b1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:02.587933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:02.588405 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:02.588509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:02.607589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:02.608759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:02.667115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1099/2211] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:02.678362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:02.680024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:02.743590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22615fd9-6c4a-4a27-a796-aacddf769209 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:02.747661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:02.748271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:02.748545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:02.748816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:02.759511 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:02.760963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:02.765994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:02.771452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1113/2212] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:03.054945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:03.056483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:03.068901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e684b-11a6-4e24-80ef-d7971631230a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:03.074878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:03.075624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:03.075972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:03.076293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] PATH_INFO: `/v3/projects/54e4914c827b4e8b818ee8a0d5516077` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:03.081750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=54e4914c827b4e8b818ee8a0d5516077)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:03.083670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:03.132346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54e4914c827b4e8b818ee8a0d5516077', 'request_id': 'req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.137986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54e4914c827b4e8b818ee8a0d5516077', 'request_id': 'req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.146660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54e4914c827b4e8b818ee8a0d5516077', 'request_id': 'req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.148663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:03.158509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] The token cache is being invalidate because project 54e4914c827b4e8b818ee8a0d5516077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:03.159186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.159818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.161312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1100/2213] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:03 2022] DELETE /identity/v3/projects/54e4914c827b4e8b818ee8a0d5516077 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:03.165444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:03.175698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e76acb-e8f5-49e7-9117-6d428f621be5 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:03.234131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:03.234742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:03.235019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:03.235284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] PATH_INFO: `/v3/users/69f0dcb8f1ce4cacadac9f71f59d3f4c` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:03.236395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69f0dcb8f1ce4cacadac9f71f59d3f4c)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:03.238356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:03.288239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69f0dcb8f1ce4cacadac9f71f59d3f4c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.295047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69f0dcb8f1ce4cacadac9f71f59d3f4c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.306998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69f0dcb8f1ce4cacadac9f71f59d3f4c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.315977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69f0dcb8f1ce4cacadac9f71f59d3f4c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.316713 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ba84e9d9-ab88-4b8b-be70-210c967fdb15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69f0dcb8f1ce4cacadac9f71f59d3f4c'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.323046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1114/2214] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:03 2022] DELETE /identity/v3/users/69f0dcb8f1ce4cacadac9f71f59d3f4c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:03.707425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:03.709223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:03.722650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a6ef-c5eb-4af9-869a-84d67b2b3bea admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:03.802221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:03.802864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:03.803141 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:03.803425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] PATH_INFO: `/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:03.804538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c1bdd24bed427cb1e1b48d5049d0e7)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:03.806521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:03.855403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c1bdd24bed427cb1e1b48d5049d0e7', 'request_id': 'req-3301bfd2-8368-4343-aabc-ea59907f2a47'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.862001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c1bdd24bed427cb1e1b48d5049d0e7', 'request_id': 'req-3301bfd2-8368-4343-aabc-ea59907f2a47'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.875501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c1bdd24bed427cb1e1b48d5049d0e7', 'request_id': 'req-3301bfd2-8368-4343-aabc-ea59907f2a47'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.886921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] The token cache is being invalidate because project 97c1bdd24bed427cb1e1b48d5049d0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:03.887203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.887834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-3301bfd2-8368-4343-aabc-ea59907f2a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:03.889327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1101/2215] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:03 2022] DELETE /identity/v3/projects/97c1bdd24bed427cb1e1b48d5049d0e7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:03.896816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:03.897311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:03.897396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:03.917856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:03.919068 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:03.982404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1115/2216] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:03.990230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:03.990514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:03.990783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.011262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:04.012468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:04.021699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1102/2217] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:04.026055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.036157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1db91b-1aa7-4755-b391-4621afff49c3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.042007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.042610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.042882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.043148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.043795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:04.044126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.045761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.058496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1116/2218] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:04.062995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.074999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6465b-5df3-4245-a3ee-4dc97d2f151f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.080991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.081628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.081903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.082172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.082782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.084689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.115849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1103/2219] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:04.120309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.130868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b351b-f5e4-4f6d-8bd6-58a34a262f5b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.135443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:04.192992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.193628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.193901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.194170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.194802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.196881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.219458 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1117/2220] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:04.223700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.234347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0d6f0-3838-4e46-a82d-97cbdfad44b5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.239113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.239714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.239991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.240260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.240897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.242522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.252425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1104/2221] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:04.256252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.266429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770aaee-880e-4363-a24c-cebbcc903fbc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.271651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.272245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.272517 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.272785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.273427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.275056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e74f9e-033e-4026-886e-60e9b30fc907 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.285986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1118/2222] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:04.290446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.300880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143273c-34dd-4955-aee7-18bcb09baa2c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.305825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.306469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.306745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.307019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] PATH_INFO: `/v3/projects/c3505c79947a47bf9c3493857c24807c/users/1360c045dd8b46fc9811b7c552448a17/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.321692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3505c79947a47bf9c3493857c24807c, user_id=1360c045dd8b46fc9811b7c552448a17, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.324346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.340086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1105/2223] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:04 2022] PUT /identity/v3/projects/c3505c79947a47bf9c3493857c24807c/users/1360c045dd8b46fc9811b7c552448a17/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:04.350479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.350757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.351018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.379021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] MFA Rules not processed for user `1360c045dd8b46fc9811b7c552448a17`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:04.380259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:04.445099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1119/2224] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:04.449983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.475910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:04.516831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc3638-ee44-42dc-9d11-c2677751535b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.521921 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.522526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.522801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.523074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.523682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.525701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.554417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1106/2225] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:04.558752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.570047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d5d0-472e-47b8-9fc7-42612f5b7357 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.627277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.627900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.628175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.628438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.629184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.631079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.656521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1120/2226] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:04.660783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.676943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b405-37df-4aa8-92bc-85c24c6b717e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.683593 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.684190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.684464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.684739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.685359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.687001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.697803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1107/2227] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:04.701917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.708661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:04.712608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debcef9-fef0-4376-9cdf-e835c7634ef7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.719030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.719677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.719949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.720214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] PATH_INFO: `/v3/projects/3452ece46b4644928ba239ef9e693d24/users/f3f7fe84254246e5b9c91de41ae370ff/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.735171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3452ece46b4644928ba239ef9e693d24, user_id=f3f7fe84254246e5b9c91de41ae370ff, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.737835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.752838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1121/2228] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:04 2022] PUT /identity/v3/projects/3452ece46b4644928ba239ef9e693d24/users/f3f7fe84254246e5b9c91de41ae370ff/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:04.757285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.772701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83c76d-5a6c-4d11-8154-b98bd6b0b722 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.839304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.839939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.840215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.840480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.841123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.842716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91edb1e2-614b-4476-9f76-754d5fb465a5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.853740 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1108/2229] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:04.858103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:04.869834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aebca7-2ef5-4cdd-8352-a3f766523bf8 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:04.874418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:04.875086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.875344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.875611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] PATH_INFO: `/v3/projects/3452ece46b4644928ba239ef9e693d24/users/f3f7fe84254246e5b9c91de41ae370ff/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.877508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3452ece46b4644928ba239ef9e693d24, user_id=f3f7fe84254246e5b9c91de41ae370ff, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:04.880233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:04.890813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1122/2230] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:04 2022] PUT /identity/v3/projects/3452ece46b4644928ba239ef9e693d24/users/f3f7fe84254246e5b9c91de41ae370ff/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:04.902368 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:04.902652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:04.902925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:04.934267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] MFA Rules not processed for user `f3f7fe84254246e5b9c91de41ae370ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:04.935574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:05.024375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1109/2231] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:05.034647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:05.036488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:05.095192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25056e-8e42-483d-8d0f-50a495cb8980 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:05.099266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:05.099868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:05.100142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:05.100412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:05.109919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:05.111375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:05.120899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1123/2232] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:05.202594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:05.212767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2191b0-1985-4f84-9a99-19e936cec6c4 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:05.216500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:05.217104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:05.217390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:05.217708 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:05.231002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:05.275667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:05.277029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d7991-04f8-4a6f-adae-cfa425b49acb service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:05.286444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1110/2233] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:08.105992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:08.173084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8b4cb-0edb-4a37-9271-f2285619cc05 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:08.177901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:08.178527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:08.178811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:08.179078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] PATH_INFO: `/v3/users/1360c045dd8b46fc9811b7c552448a17` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:08.180229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1360c045dd8b46fc9811b7c552448a17)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:08.182249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:08.235799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1360c045dd8b46fc9811b7c552448a17'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:08.242808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1360c045dd8b46fc9811b7c552448a17'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:08.253789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:08.255637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1360c045dd8b46fc9811b7c552448a17'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:08.265762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1360c045dd8b46fc9811b7c552448a17'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:08.266461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1360c045dd8b46fc9811b7c552448a17'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:08.272947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1124/2234] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:08 2022] DELETE /identity/v3/users/1360c045dd8b46fc9811b7c552448a17 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:08.277629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:08.278134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:08.278230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:08.297281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:08.298845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:08.359155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1111/2235] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:08.369000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:08.370561 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:08.440853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6363a24-ef49-4b04-aed2-78f9551134a1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:08.445094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:08.445748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:08.446027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:08.446292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:08.456309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:08.457793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:08.467974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1125/2236] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:08.924022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:08.936658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:08.949342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5cf4c-a563-4047-aaf0-46023cb2e141 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:08.955431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:08.955676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:08.956376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:08.956622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] PATH_INFO: `/v3/projects/c3505c79947a47bf9c3493857c24807c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:08.957801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3505c79947a47bf9c3493857c24807c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:08.959761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:09.009056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3505c79947a47bf9c3493857c24807c', 'request_id': 'req-0ce5b353-0c56-4f2c-897e-76dc4426cc00'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.014697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3505c79947a47bf9c3493857c24807c', 'request_id': 'req-0ce5b353-0c56-4f2c-897e-76dc4426cc00'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.024223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3505c79947a47bf9c3493857c24807c', 'request_id': 'req-0ce5b353-0c56-4f2c-897e-76dc4426cc00'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.035701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] The token cache is being invalidate because project c3505c79947a47bf9c3493857c24807c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:09.035986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.036628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.038248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1112/2237] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:08 2022] DELETE /identity/v3/projects/c3505c79947a47bf9c3493857c24807c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:09.042653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.053199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3be9b-4850-4dff-8969-5a7b91dc640e service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:09.115026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:09.115675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.115943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.116211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] PATH_INFO: `/v3/users/f3f7fe84254246e5b9c91de41ae370ff` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.117454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f7fe84254246e5b9c91de41ae370ff)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:09.119593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:09.170455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f7fe84254246e5b9c91de41ae370ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.178380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f7fe84254246e5b9c91de41ae370ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.190700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f7fe84254246e5b9c91de41ae370ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.199795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f7fe84254246e5b9c91de41ae370ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.200589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-c0744d7a-c6f0-4cab-8543-a93dbb725e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f7fe84254246e5b9c91de41ae370ff'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.206778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1126/2238] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:09 2022] DELETE /identity/v3/users/f3f7fe84254246e5b9c91de41ae370ff => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:09.508321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.509890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:09.525075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5b353-0c56-4f2c-897e-76dc4426cc00 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:09.564439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:09.587146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:09.587800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.588078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.588441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] PATH_INFO: `/v3/projects/3452ece46b4644928ba239ef9e693d24` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.589758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3452ece46b4644928ba239ef9e693d24)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:09.591687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:09.647857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3452ece46b4644928ba239ef9e693d24', 'request_id': 'req-0e630e67-24d2-46a9-8155-00ccb03807a2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.653919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3452ece46b4644928ba239ef9e693d24', 'request_id': 'req-0e630e67-24d2-46a9-8155-00ccb03807a2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.665153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3452ece46b4644928ba239ef9e693d24', 'request_id': 'req-0e630e67-24d2-46a9-8155-00ccb03807a2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.679554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] The token cache is being invalidate because project 3452ece46b4644928ba239ef9e693d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:09.679860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.680523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0e630e67-24d2-46a9-8155-00ccb03807a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:09.684546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1113/2239] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:09 2022] DELETE /identity/v3/projects/3452ece46b4644928ba239ef9e693d24 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:09.687590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.687876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.688145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.712129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:09.713408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:09.719480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:09.786749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1127/2240] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:09.794786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.795076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.795354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.815444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:09.816585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:09.825777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1114/2241] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:09.830471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.840181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dc5e8-1de8-4539-a400-fb65ed6d7a7c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:09.846072 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:09.846829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.847107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.847408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.851815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:09.852132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:09.854130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:09.868612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1128/2242] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:09.873193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.882971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb758470-347a-4013-9e35-05f8103443d8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:09.888201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:09.888925 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:09.889229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:09.889569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:09.890195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:09.892213 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:09.931135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1115/2243] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:09 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:09.934031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.946730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28c035-76e6-429a-b377-dfe37691b19c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:09.966804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:09.968920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.024301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.025230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.025594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.026020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.026772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.028919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.053472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c89b0-7fd2-4fad-8c3b-a2c450e7463c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.060903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.062275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.062275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.062511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] PATH_INFO: `/v3/users/a09e946d053f49fdaa11da1b63260a08` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.064007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09e946d053f49fdaa11da1b63260a08)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.065382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1129/2244] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:09 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:10.066276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.069020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.081403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1c754-3200-4f81-a5ec-f6e0f63741fe admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.087875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.088701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.089044 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.089317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.090089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.091686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.106120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1130/2245] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:05:10.111034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.123132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3caede-5c33-4b79-a61a-e5381c2f2b0a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.125975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:10.128641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09e946d053f49fdaa11da1b63260a08'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.128779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.129331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09e946d053f49fdaa11da1b63260a08'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.129892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.130292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09e946d053f49fdaa11da1b63260a08'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.130400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.135952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.136894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.138766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.145143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09e946d053f49fdaa11da1b63260a08'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.151945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1131/2246] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:05:10.152884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ff47a4e9-ac02-46eb-984c-6c2385143b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09e946d053f49fdaa11da1b63260a08'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.155929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.157717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.167290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1116/2247] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:09 2022] DELETE /identity/v3/users/a09e946d053f49fdaa11da1b63260a08 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:10.170412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa9fd-de21-489e-bc8a-d511aae99750 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.171819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.172105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.172381 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.192720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:10.194078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.228698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.229413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.229724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.229996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] PATH_INFO: `/v3/projects/5c0178d28004417a98bb154ee323b6a1/users/c556c62eb80849a397819ba55b8526e6/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.240894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:10.244778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0178d28004417a98bb154ee323b6a1, user_id=c556c62eb80849a397819ba55b8526e6, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.248134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceea1f-9b81-4212-83b4-93baaafa6a65 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.257304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1117/2248] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:10.263431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1132/2249] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:10 2022] PUT /identity/v3/projects/5c0178d28004417a98bb154ee323b6a1/users/c556c62eb80849a397819ba55b8526e6/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:10.264510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.265944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.274082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.274438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.274779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.303006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] MFA Rules not processed for user `c556c62eb80849a397819ba55b8526e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:10.304300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.322394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10b792-7c64-48eb-b35c-0edb10e08807 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.369584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.370525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.370639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1133/2250] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:10.370724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.370724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.374887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.400358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:10.434338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.435901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.438142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4f9bd-2999-40e7-9299-58342adf056f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.442995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.443608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.443886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.444185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.444805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.445660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1118/2251] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:10.446731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.480998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1134/2252] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:10.482590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.492816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a40c00-18f4-4585-aac6-c076f1a8ba09 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.554307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.554936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.555301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.555603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.556230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.558915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.584035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1119/2253] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:10.588182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.600210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822c0e6-8a1f-4589-b2c6-764dac2fee74 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.605086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.605776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.606053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.606319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.606971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.608599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.619039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1135/2254] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:10.623505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.633730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431514f-7e46-4bad-9b8c-73b8e233c4d3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.638648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.639387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.639724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.640061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] PATH_INFO: `/v3/projects/ff0d9643de044e3da040e45a6d837f5a/users/736ddb928e074087918978e9cef03614/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.655472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0d9643de044e3da040e45a6d837f5a, user_id=736ddb928e074087918978e9cef03614, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.659387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.679007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1120/2255] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:10 2022] PUT /identity/v3/projects/ff0d9643de044e3da040e45a6d837f5a/users/736ddb928e074087918978e9cef03614/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:10.683305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.696021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e5888-495e-4a12-9c7f-a2ee446ed58f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.725941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.727839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.767242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:10.777924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.779198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.779987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.780554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.781392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.783093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.793970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1136/2256] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:10.794808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.796549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.809891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9f553-3c4e-4217-9cc3-237523fe6779 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.814758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.815549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.815918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.816400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] PATH_INFO: `/v3/users/5b97116addc342b0828567c3472089ca` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.817861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b97116addc342b0828567c3472089ca)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.819910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.822762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a02c5-310e-427e-9040-4591f5f541fb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.827751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.828316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.828594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.828868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] PATH_INFO: `/v3/projects/ff864da3319b40fda1eefc641624b8a8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.830013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff864da3319b40fda1eefc641624b8a8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.831947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.870641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b97116addc342b0828567c3472089ca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.871097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:10.877812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b97116addc342b0828567c3472089ca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.884158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff864da3319b40fda1eefc641624b8a8', 'request_id': 'req-27c19072-cb1c-4079-94da-aa527a7ee0d8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.890281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff864da3319b40fda1eefc641624b8a8', 'request_id': 'req-27c19072-cb1c-4079-94da-aa527a7ee0d8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.891111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b97116addc342b0828567c3472089ca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.899880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff864da3319b40fda1eefc641624b8a8', 'request_id': 'req-27c19072-cb1c-4079-94da-aa527a7ee0d8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.903660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b97116addc342b0828567c3472089ca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.904275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b97116addc342b0828567c3472089ca'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.911737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] The token cache is being invalidate because project ff864da3319b40fda1eefc641624b8a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:10.912188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1137/2257] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:10 2022] DELETE /identity/v3/users/5b97116addc342b0828567c3472089ca => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:10.912303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.912736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-27c19072-cb1c-4079-94da-aa527a7ee0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:10.912834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:10.914130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.914252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1121/2258] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:10 2022] DELETE /identity/v3/projects/ff864da3319b40fda1eefc641624b8a8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:10.918830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc18b-224d-40ce-9898-f9f1f0df905b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.919196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc18b-224d-40ce-9898-f9f1f0df905b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.919528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc18b-224d-40ce-9898-f9f1f0df905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.939011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b1abc18b-224d-40ce-9898-f9f1f0df905b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:10.940229 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b1abc18b-224d-40ce-9898-f9f1f0df905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:10.981628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877af45-c260-4148-864c-bdd5a827e07a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:10.986241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:10.986899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:10.987174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:10.987438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] PATH_INFO: `/v3/projects/ff0d9643de044e3da040e45a6d837f5a/users/736ddb928e074087918978e9cef03614/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:10.989509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0d9643de044e3da040e45a6d837f5a, user_id=736ddb928e074087918978e9cef03614, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:10.992207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:10.998692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1122/2259] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:11.000015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b9f9-37d1-4c0c-8e9b-27c03e320264 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.000302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b9f9-37d1-4c0c-8e9b-27c03e320264 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.000578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54b9f9-37d1-4c0c-8e9b-27c03e320264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.012388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1138/2260] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:10 2022] PUT /identity/v3/projects/ff0d9643de044e3da040e45a6d837f5a/users/736ddb928e074087918978e9cef03614/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:11.013691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.015762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.021954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-fe54b9f9-37d1-4c0c-8e9b-27c03e320264 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.023134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fe54b9f9-37d1-4c0c-8e9b-27c03e320264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.077061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60a61e-0747-4c8a-ac93-11f579668bc6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.080729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.081293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.081623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.081841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.082622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1123/2261] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:11.084121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.084403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.084675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.095480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.097686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122edd34-1b4c-4efc-83e7-368e0848a8cc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.101741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:11.109466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1139/2262] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:11.110688 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.111243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.111739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.116309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] MFA Rules not processed for user `736ddb928e074087918978e9cef03614`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.118506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.133499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.134834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.145361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1140/2263] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:11.150071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.163979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474370a4-fc49-47c5-b5c2-5d35508a7b18 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.177836 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.178478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.181638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.181638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.181638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:11.181638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.183454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.202758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1141/2264] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:11.204785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.218608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094271c-6de3-4747-85cd-70dd8a2d7bd0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.225965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.226935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.227214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.227490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.228128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.229520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:11.230793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.254328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1124/2265] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:11.268336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1142/2266] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:11.269693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.269896 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.271395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.342454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4ddb9-88fe-44bf-8585-e7120a7a9182 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.347207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb62b1b-ada0-4219-b614-60105d06edaf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.353660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.354306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.354592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.354862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.355493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.357476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.388131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1143/2267] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:11.392874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.400204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.400913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.401271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.401566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.405855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f84189-f1a3-40ae-bd8e-a39265b33bf1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.422229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.423197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.423537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.424099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.424369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.430086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.452401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1144/2268] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:11.456529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.482580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32ee3f-d3bf-42a8-bbc6-0d972b557304 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.488649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.489266 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.489570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.489915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.491668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.494716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21effeed-9bab-4fcf-82c4-70cb76875af0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.495009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.496903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.507607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1125/2269] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:11.512949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1145/2270] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:11.514395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.526241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32970362-b261-4c53-aa3a-1f4dbcf4e088 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.531881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.532554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.532834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.533108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] PATH_INFO: `/v3/projects/01216ab86b9c455db8c4c387ec6888c4/users/174eebf90c954484b34798303ecf837b/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.550965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01216ab86b9c455db8c4c387ec6888c4, user_id=174eebf90c954484b34798303ecf837b, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.553832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.573756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1126/2271] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:11 2022] PUT /identity/v3/projects/01216ab86b9c455db8c4c387ec6888c4/users/174eebf90c954484b34798303ecf837b/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:11.584336 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.584697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.584960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.603835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.605914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.630847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] MFA Rules not processed for user `174eebf90c954484b34798303ecf837b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.632309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.647293 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:11.683330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eadb2-a7b6-4197-a17f-0a31a55c4e5e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.689219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.690057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.690391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.690736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] PATH_INFO: `/v3/projects/609a52d56a4f4aaeb50a3790f4f1edc4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.691959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609a52d56a4f4aaeb50a3790f4f1edc4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.693965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.717977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1146/2272] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:11.720365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.735509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c79c21-e84e-4f95-ac0a-468d17bee040 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.750330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:11.751187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609a52d56a4f4aaeb50a3790f4f1edc4', 'request_id': 'req-2708da33-3db7-42a6-8000-19f48d1675b4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:11.758924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609a52d56a4f4aaeb50a3790f4f1edc4', 'request_id': 'req-2708da33-3db7-42a6-8000-19f48d1675b4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:11.768463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609a52d56a4f4aaeb50a3790f4f1edc4', 'request_id': 'req-2708da33-3db7-42a6-8000-19f48d1675b4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:11.780992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] The token cache is being invalidate because project 609a52d56a4f4aaeb50a3790f4f1edc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:11.781371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:11.782150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2708da33-3db7-42a6-8000-19f48d1675b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:11.783748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1127/2273] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:11 2022] DELETE /identity/v3/projects/609a52d56a4f4aaeb50a3790f4f1edc4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:11.788595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e3bb-2985-4ee1-a261-3063c83dc4db None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.788880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e3bb-2985-4ee1-a261-3063c83dc4db None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.789402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12e3bb-2985-4ee1-a261-3063c83dc4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.809710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6e12e3bb-2985-4ee1-a261-3063c83dc4db None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.810778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.810933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6e12e3bb-2985-4ee1-a261-3063c83dc4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.812002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.812372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.812890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.813845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.815902 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.819219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1128/2274] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:11.828484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.828771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.829045 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.851184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:11.852434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.855211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1147/2275] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:11.859966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.864583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:11.936073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635bd52-54d9-43db-b921-3411b0c00da1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.940972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1129/2276] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:11.941441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.942117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.942421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.942692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.943326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.944959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.945766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.958796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5860b8-f1c8-48d8-8a48-f7955ccea905 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.964157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:11.965026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:11.965442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:11.965810 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:11.966951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:11.967455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:11.969177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:11.972274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1148/2277] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:11.976180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:11.983347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1130/2278] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:05:11.987986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859500-33a6-4489-a4b5-cd720bd7f5c3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:11.988317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.000005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.000456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12236d9-53f6-4e46-b5f6-0e9355903a59 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.001241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.001750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.002183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.003303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.005269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.005738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.006933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.007211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.007482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.008116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.010423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.021927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1149/2279] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:12.026193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.038230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c91272-f86c-4729-9f71-3c4dfba8ec73 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.043972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.044889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.045233 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.045758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.046519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.048525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.055798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1131/2280] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:11 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:12.061642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1150/2281] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:05:12.062077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.063452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.075921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d98cb6-ab26-499c-8acd-c8e49d818af0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.095453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:12.145340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.146075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.146379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.146653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] PATH_INFO: `/v3/projects/dbb3fff2748b437e834a1364f450022b/users/f9ff63aa2d354483ba7cf12bb912e9a9/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.148321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051e324-f02f-4c38-a513-2a2c84c84f2d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.154672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.155295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.155569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.155838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.156466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.158407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.161334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:12.166331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb3fff2748b437e834a1364f450022b, user_id=f9ff63aa2d354483ba7cf12bb912e9a9, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.172023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0845e4-2590-44f5-99b7-5579f2c0975a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.188567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1132/2282] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:12.192996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.197653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1151/2283] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:12 2022] PUT /identity/v3/projects/dbb3fff2748b437e834a1364f450022b/users/f9ff63aa2d354483ba7cf12bb912e9a9/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:12.217528 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4db3d3-48c4-47d9-a2fc-22ef55c20bd5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.268263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.268543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.268812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.294869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.295739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.296155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.296530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.297311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.299525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.304145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] MFA Rules not processed for user `f9ff63aa2d354483ba7cf12bb912e9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:12.305392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:12.310758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1133/2284] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:05:12.315110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.326010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe5558-72ff-4706-bc4f-ea22f6b927a0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.330987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.331911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.331911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.332182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.332918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.334549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.346097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1134/2285] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:12.351129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.361860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382bd0-8f95-420a-bae2-35513edd9c8d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.366689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.367422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.367846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.368126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] PATH_INFO: `/v3/projects/efac6ddeea0d4d458b7de89c533e540d/users/1fa6eb4bc1e54b6a80b038ea768d902c/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.383902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efac6ddeea0d4d458b7de89c533e540d, user_id=1fa6eb4bc1e54b6a80b038ea768d902c, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.387025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc383c-5d11-4e9b-8950-59e9a23137d8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.401577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1152/2286] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:12.406052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1135/2287] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:12 2022] PUT /identity/v3/projects/efac6ddeea0d4d458b7de89c533e540d/users/1fa6eb4bc1e54b6a80b038ea768d902c/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:12.406188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.407610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.407991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.408331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.415889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f87d55-6533-4fbf-94e7-47f5e6192df4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.427947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:12.429400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:12.493573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.493573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.493573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.494095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.494644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.497180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.548723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1136/2288] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:12.557058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.558676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:12.575942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1153/2289] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:12.577686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.577832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:12.642694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9901ad5-e477-40b8-9a06-9253da6d7325 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.660401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910213ed-6d2e-4dd7-a4ce-c4ddd44d98e1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.666120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.666998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.667440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.667846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.669102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.671966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.683816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:12.709523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1154/2290] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:12.714970 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.724527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.725180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.725499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.725755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.727504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf029224-5116-4a89-ae04-5206bb608775 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.732969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.733814 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.734146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.734652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.735389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.737124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.741938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.743400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.752892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1155/2291] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:12.755354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1137/2292] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:05:12.756739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.768758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19997d8-45fc-4146-91d2-106098ed0f0b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.773980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.774642 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.774914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.775177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] PATH_INFO: `/v3/projects/50a5ade55d054eec939e0cf409428d6d/users/6e14d86aeea84221810311a85b11c1b6/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.798745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a5ade55d054eec939e0cf409428d6d, user_id=6e14d86aeea84221810311a85b11c1b6, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.801462 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.819808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1156/2293] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:12 2022] PUT /identity/v3/projects/50a5ade55d054eec939e0cf409428d6d/users/6e14d86aeea84221810311a85b11c1b6/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:12.824202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.838373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59b938-f634-4140-8950-384a98339272 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.910101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.910878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.911216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.911497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.912197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.914023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbd4d5-34af-44ce-96d7-6f01daa39019 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.925886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1138/2294] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:12.930156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:12.941271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990c221-38c8-455c-922c-94802dfb2dcd admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:12.946481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:12.947008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.947603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.947603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] PATH_INFO: `/v3/projects/50a5ade55d054eec939e0cf409428d6d/users/6e14d86aeea84221810311a85b11c1b6/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:12.949610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a5ade55d054eec939e0cf409428d6d, user_id=6e14d86aeea84221810311a85b11c1b6, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:12.952609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:12.963639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1157/2295] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:12 2022] PUT /identity/v3/projects/50a5ade55d054eec939e0cf409428d6d/users/6e14d86aeea84221810311a85b11c1b6/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:12.975041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:12.975431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:12.975779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:13.007262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] MFA Rules not processed for user `6e14d86aeea84221810311a85b11c1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:13.008719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.071879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:13.098118 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1139/2296] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:13.111677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:13.114466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.187730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88153cf-9a04-4c30-922a-d15674540c76 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:13.191851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:13.192669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:13.193019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:13.193263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:13.195768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:13.197265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.198870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.273387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:13.274903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c9490-9004-4271-b230-8173d44b4c82 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:13.275020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99b71d-9673-4c90-b807-fb05b53134a1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:13.280119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:13.280802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:13.281078 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:13.281351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] PATH_INFO: `/v3/users/c556c62eb80849a397819ba55b8526e6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:13.282524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c556c62eb80849a397819ba55b8526e6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:13.284455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:13.285287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1158/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:13.345031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c556c62eb80849a397819ba55b8526e6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:13.345739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c556c62eb80849a397819ba55b8526e6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:13.346383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c556c62eb80849a397819ba55b8526e6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:13.362347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c556c62eb80849a397819ba55b8526e6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:13.371558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c556c62eb80849a397819ba55b8526e6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:13.387487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1140/2298] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:13 2022] DELETE /identity/v3/users/c556c62eb80849a397819ba55b8526e6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:13.393059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e551f2-dded-400a-bdd0-2d0c71f469a0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:13.393344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e551f2-dded-400a-bdd0-2d0c71f469a0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:13.393759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e551f2-dded-400a-bdd0-2d0c71f469a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:13.414077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-65e551f2-dded-400a-bdd0-2d0c71f469a0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:13.415473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-65e551f2-dded-400a-bdd0-2d0c71f469a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.478354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:13.494086 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1159/2299] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:13.504062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:13.505335 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:13.566053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f975d5d1-c5c9-4473-b5fb-d4f8f5c28737 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:13.570785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:13.571900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:13.572466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:13.572951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:13.584873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:13.586357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:13.596906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1141/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:13.996191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:13.996491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:13.997273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:14.045945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:14.047431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:14.054542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] MFA Rules not processed for user `1fa6eb4bc1e54b6a80b038ea768d902c`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:14.055782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:14.070336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a5155-c157-4939-a799-580f825145f0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:14.082267 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:14.092939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:14.093511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:14.094013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] PATH_INFO: `/v3/projects/5c0178d28004417a98bb154ee323b6a1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:14.096643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0178d28004417a98bb154ee323b6a1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:14.101219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:14.118369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:14.155890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1160/2301] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:14.174816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:14.178034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:14.183656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0178d28004417a98bb154ee323b6a1', 'request_id': 'req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.215278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0178d28004417a98bb154ee323b6a1', 'request_id': 'req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.237162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0178d28004417a98bb154ee323b6a1', 'request_id': 'req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.250939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4e65a-eb73-4da9-82db-ff73f55cbc1b None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:14.256663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] The token cache is being invalidate because project 5c0178d28004417a98bb154ee323b6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:14.257441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.258591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.261527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1142/2302] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:14 2022] DELETE /identity/v3/projects/5c0178d28004417a98bb154ee323b6a1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:14.265641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:14.267722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:14.322416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:14.323252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:14.323534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:14.323804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:14.325697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:14.370940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bdb3e-b2f3-42fb-a998-bf3ccc9713bc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:14.377383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:14.378220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:14.378358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:14.378676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] PATH_INFO: `/v3/users/736ddb928e074087918978e9cef03614` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:14.379840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=736ddb928e074087918978e9cef03614)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:14.382261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:14.409400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:14.411009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:14.416169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:14.421835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1161/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:05:14.441893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736ddb928e074087918978e9cef03614'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.442823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736ddb928e074087918978e9cef03614'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.444002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736ddb928e074087918978e9cef03614'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.483056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736ddb928e074087918978e9cef03614'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.497610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736ddb928e074087918978e9cef03614'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:14.516961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1143/2304] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:14 2022] DELETE /identity/v3/users/736ddb928e074087918978e9cef03614 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:14.890087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:14.904321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc224567-b622-4f62-9492-139c4859fd9c service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.024442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.027472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.049280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.050023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.050246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.050516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] PATH_INFO: `/v3/projects/ff0d9643de044e3da040e45a6d837f5a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.051751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0d9643de044e3da040e45a6d837f5a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.053629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:15.053754 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.103455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87558-856e-4ff4-af66-3468eda95fec admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.107538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.108437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.108923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.109514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.111799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.126666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0d9643de044e3da040e45a6d837f5a', 'request_id': 'req-d869660b-3894-40d8-8595-5c1ed56a6c31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:15.135419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0d9643de044e3da040e45a6d837f5a', 'request_id': 'req-d869660b-3894-40d8-8595-5c1ed56a6c31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:15.154716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0d9643de044e3da040e45a6d837f5a', 'request_id': 'req-d869660b-3894-40d8-8595-5c1ed56a6c31'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:15.179082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] The token cache is being invalidate because project ff0d9643de044e3da040e45a6d837f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:15.179378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:15.181927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d869660b-3894-40d8-8595-5c1ed56a6c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:15.184535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1162/2305] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:14 2022] DELETE /identity/v3/projects/ff0d9643de044e3da040e45a6d837f5a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:15.189607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.190245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d675-df63-4828-980d-7764b32018cc None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.190609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d675-df63-4828-980d-7764b32018cc None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.190924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d675-df63-4828-980d-7764b32018cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.191350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d30e5f-115c-466e-abd7-c1cec4de9b0b service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.201796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1144/2306] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:15.215152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-28f8d675-df63-4828-980d-7764b32018cc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:15.216473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-28f8d675-df63-4828-980d-7764b32018cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.289023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1163/2307] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:15.299055 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.299369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.299649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.323791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:15.325124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.348939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1145/2308] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:15.351757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8d675-df63-4828-980d-7764b32018cc None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.361915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8d675-df63-4828-980d-7764b32018cc None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.366669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.367286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.367565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.367837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.368503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:15.368807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.370534 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.385681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1164/2309] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:15.387785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.401044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1652c9-3f19-43a6-85c0-3c79dc401a53 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.406114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.406806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.407150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.407509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.408318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.410263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.442132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1146/2310] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:15.454893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.465711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48262622-4913-4651-b9f2-13c19493b1b1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.479285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:15.531142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.531759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.532033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.532302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.532930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.534926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.564219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1165/2311] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:15.568480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.580509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695c6a7-bdd7-418f-bcbe-3888372678eb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.588724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.589480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.589779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.590053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.590758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.593786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.609146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1147/2312] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:15.613441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.627410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8023-1e32-4d31-be54-29cd2fa71ac5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.635529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.636196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.636420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.636693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.637353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.639177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d4fae-15c4-4e3a-8ebe-e75076b523da admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.653981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1166/2313] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:15.657577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.671588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb5564-b047-4d9f-abd1-04031d5efd93 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:15.678450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:15.679600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.680032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.680399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] PATH_INFO: `/v3/projects/b00d94096e4345c8b83df79d9bb82a9d/users/1b0e155aa68f42f09e33083e669b0b2f/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.723806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00d94096e4345c8b83df79d9bb82a9d, user_id=1b0e155aa68f42f09e33083e669b0b2f, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:15.727389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378af10-3079-45b1-ab63-91bd3b603863 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:15.751792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1148/2314] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:15 2022] PUT /identity/v3/projects/b00d94096e4345c8b83df79d9bb82a9d/users/1b0e155aa68f42f09e33083e669b0b2f/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:15.768071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.768352 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.768624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.817104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] MFA Rules not processed for user `1b0e155aa68f42f09e33083e669b0b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:15.818335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.907282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1167/2315] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:15.931545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:15.931823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:15.932092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:15.944735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:15.947457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:15.954327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] MFA Rules not processed for user `1fa6eb4bc1e54b6a80b038ea768d902c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:15.955532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.018184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea220ec-0e0c-4d42-ad63-4a12dcb04acd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:16.021923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:16.022539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.022808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.023109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.024856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.029530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1149/2316] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:16.078383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:16.080743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.089108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:16.098500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e066d-0a56-4658-a810-26b707f881f0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:16.102497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:16.103178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.104216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.104913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.113531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:16.114028 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:16.115176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f213672-da02-42ce-b78c-172bc5c4b8f9 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:16.115667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:16.119110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:16.127625 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1168/2317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:16.128395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1150/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:05:16.144712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:16.217238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fedc80-c8a3-4131-a868-07122d3cd898 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:16.224565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:16.225443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.226207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.226769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] PATH_INFO: `/v3/users/1b0e155aa68f42f09e33083e669b0b2f` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.228326 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0e155aa68f42f09e33083e669b0b2f)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:16.234270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:16.287344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0e155aa68f42f09e33083e669b0b2f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.294830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0e155aa68f42f09e33083e669b0b2f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.307425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0e155aa68f42f09e33083e669b0b2f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.317136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0e155aa68f42f09e33083e669b0b2f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.318146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0e155aa68f42f09e33083e669b0b2f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.324321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1169/2319] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:16 2022] DELETE /identity/v3/users/1b0e155aa68f42f09e33083e669b0b2f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:16.328947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.329299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.329698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.348782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:16.350217 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.411175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1151/2320] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:16.421111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:16.422658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.481394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887d716-23be-4816-9948-303df38d1003 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:16.485070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:16.485702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.485973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.486236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.495503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:16.496935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f018df-9044-43fe-8bf7-0c7584ce9751 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:16.500827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:16.508651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1170/2321] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:05:16.797631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:16.799650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:16.815865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d8b0c-cf9d-43ad-9899-a556413fac0f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:16.824526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:16.825206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.825488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.825801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] PATH_INFO: `/v3/projects/b00d94096e4345c8b83df79d9bb82a9d` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.827029 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00d94096e4345c8b83df79d9bb82a9d)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:16.828994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:16.899504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00d94096e4345c8b83df79d9bb82a9d', 'request_id': 'req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.906391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00d94096e4345c8b83df79d9bb82a9d', 'request_id': 'req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.915743 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00d94096e4345c8b83df79d9bb82a9d', 'request_id': 'req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.929231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] The token cache is being invalidate because project b00d94096e4345c8b83df79d9bb82a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:16.929523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.930212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ef7f5cc0-6b80-4294-9dd7-2e10411f571c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:16.932081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1152/2322] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:16 2022] DELETE /identity/v3/projects/b00d94096e4345c8b83df79d9bb82a9d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:16.937515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:16.937856 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:16.938123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:16.959710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:16.960932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:17.022692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1171/2323] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:17.030860 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.031330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.031434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.053022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:17.054353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:17.063180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1153/2324] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:17.067522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.077565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fadb59-5e05-4b3c-9ab2-70920ccda864 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.082151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.082767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.083019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.083282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.083923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:17.084218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.085865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.099060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1172/2325] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:17.103571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.114197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275e5f-bbd6-44a3-9120-31c40aaa84e0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.118776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.119375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.119650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.119940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.120570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.122506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.158664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1154/2326] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:17 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:17.160584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.171165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d62c3-482d-4760-bd7a-198ce3d57162 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.231643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.232286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.232544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.232809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.233459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.235441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.260029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1173/2327] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:17 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:05:17.265946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.276595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5756-4a74-4703-9f1f-900b1844f18c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.282003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.282604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.282897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.283197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.283796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.285613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.296802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1155/2328] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:17.300993 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.311948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bbfc-93ca-4b37-9ce9-d57dc171b549 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.316988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.317590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.317889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.318155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.318798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.320395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47248a-4ad0-4d99-83d3-bb3ca869d700 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.330901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1174/2329] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:17.335121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.345690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f7ff-de8f-4ebc-a4a2-0b8c98dda894 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.350730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.351377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.351657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.352621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] PATH_INFO: `/v3/projects/37e47abc252f4822a419a00eac43add8/users/60665776cb13486a8792622ced323493/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.367342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e47abc252f4822a419a00eac43add8, user_id=60665776cb13486a8792622ced323493, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.370126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.384927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1156/2330] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:17 2022] PUT /identity/v3/projects/37e47abc252f4822a419a00eac43add8/users/60665776cb13486a8792622ced323493/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:17.395364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960eee-9208-4d97-87ce-ae8e291506eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.395615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960eee-9208-4d97-87ce-ae8e291506eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.395879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960eee-9208-4d97-87ce-ae8e291506eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.425615 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-33960eee-9208-4d97-87ce-ae8e291506eb None None] MFA Rules not processed for user `60665776cb13486a8792622ced323493`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:17.426747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-33960eee-9208-4d97-87ce-ae8e291506eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:17.489587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:17.501843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1175/2331] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:17.513744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:17.515612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:17.535804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:17.575298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1696d-1f98-491d-b783-d8e7dce407db admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:17.579410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:17.580022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:17.580300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:17.580571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:17.589357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:17.590845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:17.599755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1157/2332] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:21.823197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7b39-dc43-46c9-a24e-9ec0fb3feadd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:21.823746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7b39-dc43-46c9-a24e-9ec0fb3feadd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:21.823746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca7b39-dc43-46c9-a24e-9ec0fb3feadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:21.846386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-58ca7b39-dc43-46c9-a24e-9ec0fb3feadd None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:21.847571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-58ca7b39-dc43-46c9-a24e-9ec0fb3feadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:21.905683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1176/2333] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:05:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:21.929181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:21.930776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:22.000144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919de1d-e3ff-41c1-9bd7-20e032572153 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:22.006758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:22.007380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:22.007662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:22.007934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:22.016869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:22.018418 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:22.026746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1158/2334] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:05:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:05:30.847842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887281-1568-46c3-9a99-a75acfdfb04e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:30.848818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887281-1568-46c3-9a99-a75acfdfb04e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:30.849092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa887281-1568-46c3-9a99-a75acfdfb04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:30.870778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fa887281-1568-46c3-9a99-a75acfdfb04e None None] MFA Rules not processed for user `a5aece8a7f2342cfabd9a3f8707a4470`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:30.872525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fa887281-1568-46c3-9a99-a75acfdfb04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:30.946342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1177/2335] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:32.023425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=50 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:34.347477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:34.349341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:34.427005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfbee4-8047-4922-9f24-993579d843f0 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:34.433282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:34.433976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:34.434249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:34.434518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] PATH_INFO: `/v3/users/174eebf90c954484b34798303ecf837b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:34.435655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] RBAC: Authorizing `identity:delete_user(user_id=174eebf90c954484b34798303ecf837b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:34.437659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:34.492898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174eebf90c954484b34798303ecf837b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:34.493585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174eebf90c954484b34798303ecf837b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:34.494445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174eebf90c954484b34798303ecf837b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:34.507463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174eebf90c954484b34798303ecf837b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:34.515872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174eebf90c954484b34798303ecf837b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:34.533889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1159/2336] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:34 2022] DELETE /identity/v3/users/174eebf90c954484b34798303ecf837b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:34.539270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:34.539799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:34.539799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:34.561674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:34.562964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:34.645291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1178/2337] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:34.655106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:34.656660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:34.724913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e64ae-1c76-46fa-8758-663d3c7ca60a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:34.729482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:34.730193 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:34.730565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:34.730914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:34.741781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:34.743344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:34.753629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1160/2338] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:35.041606 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:35.043833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:35.070906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94d1e-4b1c-4e2f-8a0b-88417b06cb13 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:35.078442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:35.079306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:35.079461 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:35.079730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] PATH_INFO: `/v3/projects/01216ab86b9c455db8c4c387ec6888c4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:35.081413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01216ab86b9c455db8c4c387ec6888c4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:35.083639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:35.130815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:35.142205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01216ab86b9c455db8c4c387ec6888c4', 'request_id': 'req-fc73bc45-7850-4285-9505-8b4e21135334'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.155052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01216ab86b9c455db8c4c387ec6888c4', 'request_id': 'req-fc73bc45-7850-4285-9505-8b4e21135334'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.162568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01216ab86b9c455db8c4c387ec6888c4', 'request_id': 'req-fc73bc45-7850-4285-9505-8b4e21135334'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.177427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] The token cache is being invalidate because project 01216ab86b9c455db8c4c387ec6888c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:35.178020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.178687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.180465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1179/2339] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:35 2022] DELETE /identity/v3/projects/01216ab86b9c455db8c4c387ec6888c4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:35.184801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:35.195836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4bfa2-9d14-49c2-ab1f-e40bf2c743f8 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:35.213929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:35.264359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:35.265031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:35.265297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:35.265626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] PATH_INFO: `/v3/users/f9ff63aa2d354483ba7cf12bb912e9a9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:35.266877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ff63aa2d354483ba7cf12bb912e9a9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:35.268764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:35.320582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ff63aa2d354483ba7cf12bb912e9a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.321224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ff63aa2d354483ba7cf12bb912e9a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.321906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ff63aa2d354483ba7cf12bb912e9a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.334792 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ff63aa2d354483ba7cf12bb912e9a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.343285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ff63aa2d354483ba7cf12bb912e9a9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:35.361706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1161/2340] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:35 2022] DELETE /identity/v3/users/f9ff63aa2d354483ba7cf12bb912e9a9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:36.015281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:36.017640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:36.036308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73bc45-7850-4285-9505-8b4e21135334 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:36.163084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:36.163759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:36.164029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:36.164294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] PATH_INFO: `/v3/projects/dbb3fff2748b437e834a1364f450022b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:36.165933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb3fff2748b437e834a1364f450022b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:36.168181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:36.233401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb3fff2748b437e834a1364f450022b', 'request_id': 'req-110dd955-b6bb-4683-a706-0c4b78fa5615'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.244076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb3fff2748b437e834a1364f450022b', 'request_id': 'req-110dd955-b6bb-4683-a706-0c4b78fa5615'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.251023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb3fff2748b437e834a1364f450022b', 'request_id': 'req-110dd955-b6bb-4683-a706-0c4b78fa5615'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.272717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] The token cache is being invalidate because project dbb3fff2748b437e834a1364f450022b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:36.273215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.274714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.279335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1180/2341] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:36 2022] DELETE /identity/v3/projects/dbb3fff2748b437e834a1364f450022b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:36.280699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:36.287162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:36.305711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53cfd1-5218-41f7-aae2-bf1bc121d2e5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:36.387091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:36.388563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:36.389009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:36.389444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] PATH_INFO: `/v3/users/6e14d86aeea84221810311a85b11c1b6` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:36.390777 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e14d86aeea84221810311a85b11c1b6)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:36.392882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:36.472035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:36.474673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e14d86aeea84221810311a85b11c1b6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.475732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e14d86aeea84221810311a85b11c1b6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.476643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e14d86aeea84221810311a85b11c1b6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.499592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e14d86aeea84221810311a85b11c1b6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.507500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ab7574-d6bc-4b35-8ac6-2b323460adff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e14d86aeea84221810311a85b11c1b6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:36.525075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1162/2342] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:36 2022] DELETE /identity/v3/users/6e14d86aeea84221810311a85b11c1b6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:36.861563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:36.863056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:36.894961 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dd955-b6bb-4683-a706-0c4b78fa5615 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:36.999502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:37.007005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:37.007742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:37.008490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:37.008931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] PATH_INFO: `/v3/projects/50a5ade55d054eec939e0cf409428d6d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:37.010299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a5ade55d054eec939e0cf409428d6d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:37.012565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:37.126183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a5ade55d054eec939e0cf409428d6d', 'request_id': 'req-b2708516-705b-422d-b8e1-bfa97bb9b2db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:37.134545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a5ade55d054eec939e0cf409428d6d', 'request_id': 'req-b2708516-705b-422d-b8e1-bfa97bb9b2db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:37.146197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a5ade55d054eec939e0cf409428d6d', 'request_id': 'req-b2708516-705b-422d-b8e1-bfa97bb9b2db'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:37.169439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] The token cache is being invalidate because project 50a5ade55d054eec939e0cf409428d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:37.169903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:37.170719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:37.185568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1181/2343] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:36 2022] DELETE /identity/v3/projects/50a5ade55d054eec939e0cf409428d6d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:43.997020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:43.997926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:43.998344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:44.020134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:44.021630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:44.107712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:44.109469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:44.110736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1163/2344] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:44.247438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708516-705b-422d-b8e1-bfa97bb9b2db admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:44.266246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:44.267099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:44.267571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:44.268067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:44.290730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:44.292931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:44.302726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1182/2345] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:05:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 20:05:46.314009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:46.315559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:46.383537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111b2e6-6816-4882-9d99-addb5e89f74e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:46.388790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:46.389422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:46.389835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:46.390116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] PATH_INFO: `/v3/users/1e6508646820461591015314de163c2f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:46.391296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6508646820461591015314de163c2f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:46.393205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:46.446941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6508646820461591015314de163c2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.447688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6508646820461591015314de163c2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.448363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6508646820461591015314de163c2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.460074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6508646820461591015314de163c2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.462124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:46.468627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a3371acb-aade-40f9-bb70-abb3fd6f361a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6508646820461591015314de163c2f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.484552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1164/2346] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:46 2022] DELETE /identity/v3/users/1e6508646820461591015314de163c2f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:46.678749 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:46.714832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaa3d4-57dc-4d99-b17c-cb925c8af7c7 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:46.788351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:46.788989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:46.789263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:46.789547 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] PATH_INFO: `/v3/projects/04042df62e7941d7a6d9d1cf493d9602` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:46.791097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=04042df62e7941d7a6d9d1cf493d9602)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:46.793031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:46.815272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:46.845252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04042df62e7941d7a6d9d1cf493d9602', 'request_id': 'req-28bb17fd-e414-4d87-8ed9-5740dac084ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.850763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04042df62e7941d7a6d9d1cf493d9602', 'request_id': 'req-28bb17fd-e414-4d87-8ed9-5740dac084ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.856956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04042df62e7941d7a6d9d1cf493d9602', 'request_id': 'req-28bb17fd-e414-4d87-8ed9-5740dac084ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.873333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] The token cache is being invalidate because project 04042df62e7941d7a6d9d1cf493d9602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:46.873636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.874278 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-28bb17fd-e414-4d87-8ed9-5740dac084ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:46.875762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1183/2347] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:46 2022] DELETE /identity/v3/projects/04042df62e7941d7a6d9d1cf493d9602 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:46.881723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:46.882011 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:46.882279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:46.903996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:46.905327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:46.973119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1165/2348] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:46.981228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:46.981508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:46.981817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.004550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:47.006033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:47.014045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1184/2349] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:47.018312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.028330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114528c-ae59-4774-ac77-6e3f5def482c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.033711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.034324 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.034585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.034854 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.035502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:47.035799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.037455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.050188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1166/2350] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:47.054675 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.064025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e576289-8a78-4a94-b31d-bf7e7bbbc177 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.068602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.069196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.069466 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.069767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.070388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.072296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.101363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1185/2351] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:47.105813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.116350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97d8a1-ff83-486d-89d5-dd11e2b2a224 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.172897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.173878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.174151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.174416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.175025 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.176946 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.204407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1167/2352] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:47 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:47.208733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.219509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690b7c6-0fb2-4dd2-bf48-a4902036c5cf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.224766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.225359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.225656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.225921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.226559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.228152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.238679 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1186/2353] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:47.242637 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.253281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c198c9-c507-4bd9-972d-f1a5a65d3101 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.258852 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.259450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.259725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.259991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.260689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.262316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688cf11-8021-426b-8499-3588e729e5cb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.274302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1168/2354] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:47.278702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.294269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e9ca7-08d5-425f-a8ba-64c812e5b24a admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.302881 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.303840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.304228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.304604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] PATH_INFO: `/v3/projects/a6342015b474443a9d2ba1f9a302c227/users/b222efbaed7e4e90aa51b3e1eeb406c5/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.320816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6342015b474443a9d2ba1f9a302c227, user_id=b222efbaed7e4e90aa51b3e1eeb406c5, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.323489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.338280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1187/2355] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:47 2022] PUT /identity/v3/projects/a6342015b474443a9d2ba1f9a302c227/users/b222efbaed7e4e90aa51b3e1eeb406c5/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:47.349275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664666f1-7394-4024-be94-a41ae994247f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.349808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664666f1-7394-4024-be94-a41ae994247f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.349934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664666f1-7394-4024-be94-a41ae994247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.379632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-664666f1-7394-4024-be94-a41ae994247f None None] MFA Rules not processed for user `b222efbaed7e4e90aa51b3e1eeb406c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:47.380877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-664666f1-7394-4024-be94-a41ae994247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:47.425769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:47.458248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1169/2356] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:47.470002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.472193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:47.541581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c31b45-1045-4fee-882f-532d71dc2ef0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:47.545516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:47.546121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:47.546391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:47.546651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:47.555658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:47.557141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764fb465-2bcd-4445-9a8e-ae43769a7582 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:47.566302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1188/2357] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:47.936271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664666f1-7394-4024-be94-a41ae994247f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:47.937762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-664666f1-7394-4024-be94-a41ae994247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:48.006533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664666f1-7394-4024-be94-a41ae994247f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:48.012043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:48.012690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:48.013083 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:48.013382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] PATH_INFO: `/v3/users/60665776cb13486a8792622ced323493` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:48.014590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=60665776cb13486a8792622ced323493)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:48.016562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:48.065673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60665776cb13486a8792622ced323493'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.066285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60665776cb13486a8792622ced323493'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.066937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60665776cb13486a8792622ced323493'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.078589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60665776cb13486a8792622ced323493'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.085933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60665776cb13486a8792622ced323493'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.100641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1170/2358] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:47 2022] DELETE /identity/v3/users/60665776cb13486a8792622ced323493 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:48.105301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:48.105829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:48.105922 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:48.125644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:48.126850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:48.170657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:48.191216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1189/2359] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:48.201233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:48.202723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:48.259327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05918de4-8d06-4d5f-9d33-965b999bb7dc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:48.263100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:48.263710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:48.264014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:48.264284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:48.273596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:48.275082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:48.284424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1171/2360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:05:48.570982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:48.571752 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:48.584528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc968d-fde8-4d9e-b850-a56e956d0f33 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:48.591358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:48.592006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:48.592282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:48.592555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] PATH_INFO: `/v3/projects/37e47abc252f4822a419a00eac43add8` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:48.593720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e47abc252f4822a419a00eac43add8)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:48.595831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:48.649423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e47abc252f4822a419a00eac43add8', 'request_id': 'req-607bf454-8a39-4945-9544-4d6d215cf342'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.655088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e47abc252f4822a419a00eac43add8', 'request_id': 'req-607bf454-8a39-4945-9544-4d6d215cf342'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.661284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e47abc252f4822a419a00eac43add8', 'request_id': 'req-607bf454-8a39-4945-9544-4d6d215cf342'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.679847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] The token cache is being invalidate because project 37e47abc252f4822a419a00eac43add8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:48.680159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.680894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-607bf454-8a39-4945-9544-4d6d215cf342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:48.683228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1190/2361] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:48 2022] DELETE /identity/v3/projects/37e47abc252f4822a419a00eac43add8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:50.179949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:50.181576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:50.189952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:50.195923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd52a5a-e542-4638-9650-a6ae1d4fb883 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:50.267227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:50.267957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:50.268237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:50.268519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] PATH_INFO: `/v3/users/b222efbaed7e4e90aa51b3e1eeb406c5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:50.269744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b222efbaed7e4e90aa51b3e1eeb406c5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:50.271707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:50.328172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b222efbaed7e4e90aa51b3e1eeb406c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:50.329035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b222efbaed7e4e90aa51b3e1eeb406c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:50.329795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b222efbaed7e4e90aa51b3e1eeb406c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:50.355069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b222efbaed7e4e90aa51b3e1eeb406c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:50.363231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b222efbaed7e4e90aa51b3e1eeb406c5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:50.390926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1172/2362] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:05:50 2022] DELETE /identity/v3/users/b222efbaed7e4e90aa51b3e1eeb406c5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:05:50.391169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:50.391169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:50.391169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:50.409889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:50.411109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:50.474860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1191/2363] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:50.485970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:50.487595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:50.562228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1ad85-0874-486e-ae77-087b20dfac83 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:50.566216 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:50.567128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:50.567906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:50.568273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:50.578391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:50.579908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467016e2-8699-4b3a-be51-d3361d59c0be service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:50.590453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1173/2364] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:05:50.908489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:50.909980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:50.931938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7c14f-04a4-43f7-9188-7600b4e87f2d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:50.937554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:50.938205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:50.938479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:50.938748 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] PATH_INFO: `/v3/projects/a6342015b474443a9d2ba1f9a302c227` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:50.939926 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6342015b474443a9d2ba1f9a302c227)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:50.942026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:50.996050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6342015b474443a9d2ba1f9a302c227', 'request_id': 'req-3e55413d-710a-41d7-9a43-92db2dd8ccfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:51.002998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6342015b474443a9d2ba1f9a302c227', 'request_id': 'req-3e55413d-710a-41d7-9a43-92db2dd8ccfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:51.014574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6342015b474443a9d2ba1f9a302c227', 'request_id': 'req-3e55413d-710a-41d7-9a43-92db2dd8ccfc'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:51.017434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:51.032741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] The token cache is being invalidate because project a6342015b474443a9d2ba1f9a302c227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:05:51.033036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:51.034269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3e55413d-710a-41d7-9a43-92db2dd8ccfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:05:51.035852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1192/2365] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:05:50 2022] DELETE /identity/v3/projects/a6342015b474443a9d2ba1f9a302c227 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:51.040845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.041209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.041635 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.069261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:51.070592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:51.090781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:05:51.150691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1174/2366] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:05:51.158997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.159301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.159531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.196438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:51.197737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:51.206412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1193/2367] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:51.211081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.222621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e9342-32bd-41e5-8ffc-1b5b077e7283 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.228273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.229003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.229359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.230327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.231036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:05:51.231342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.232956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.245112 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1175/2368] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:05:51.249763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.259047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d57b-79a4-49d6-bb4e-5dfea5d73c3c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.263718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.264308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.264577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.264842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.265467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.267404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.300345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1194/2369] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:05:51 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:51.305462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.316714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32df25d-78d1-4f2d-a53b-24d89b16fa47 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.379644 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.380245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.380547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.380814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.381438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.383383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.407939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1176/2370] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:05:51 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:05:51.412270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.423818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9cf5-182a-404c-a3e1-088b2a806b75 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.431328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.432253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.432529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.432792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.433454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.435102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.446262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1195/2371] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:05:51.450572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.461355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26383e24-c6a4-4755-bb3c-b7beeb89b10d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.467376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.467994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.468275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.468546 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.469203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.470843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392e41c-6d3e-45b3-bc7c-d10da752986e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.483650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1177/2372] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:05:51.488039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.498882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9ad54-5bba-4397-b21e-9d256aaa3b7d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.503948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.504629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.504912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.505183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] PATH_INFO: `/v3/projects/149c64dc002f4a49bcedd1e681e0bace/users/1755247ff27642a7a422eec3d6c5b71b/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.522673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=149c64dc002f4a49bcedd1e681e0bace, user_id=1755247ff27642a7a422eec3d6c5b71b, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.525343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.551046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1196/2373] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:05:51 2022] PUT /identity/v3/projects/149c64dc002f4a49bcedd1e681e0bace/users/1755247ff27642a7a422eec3d6c5b71b/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:05:51.561775 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186242-c100-417b-a2df-a1f8d01cb4d9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.562062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186242-c100-417b-a2df-a1f8d01cb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.562356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81186242-c100-417b-a2df-a1f8d01cb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.592156 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-81186242-c100-417b-a2df-a1f8d01cb4d9 None None] MFA Rules not processed for user `1755247ff27642a7a422eec3d6c5b71b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:05:51.593376 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-81186242-c100-417b-a2df-a1f8d01cb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:51.660270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1178/2374] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:05:51.672762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:05:51.674963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:05:51.734994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bd12a-a450-407b-93bc-31bf7361d38f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:05:51.739125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:05:51.739727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:05:51.740000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:05:51.740297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:05:51.749795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:05:51.751270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:05:51.760860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1197/2375] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:06:01.657345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:06:01.756640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:06:53.863555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:53.864084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:53.864185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:53.891466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] MFA Rules not processed for user `454c2bf24ed94f82b453df36e6e8eac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:06:53.892745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:53.950227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1179/2376] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:06:55.163203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:55.164721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:55.236509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53827de-24a7-4216-91ac-c1a357a2346c service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:55.242173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:55.242722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:55.242963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:55.243251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:55.245107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:55.256583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:55.258221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e0a8a-2768-4a76-b635-83ef4243e59e service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:55.267812 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1198/2377] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:06:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:06:58.308882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:58.398088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b478248-e351-4560-8b9d-ce13d547303c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:58.403369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:58.404065 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:58.404417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:58.404727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] PATH_INFO: `/v3/users/1755247ff27642a7a422eec3d6c5b71b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:58.406711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1755247ff27642a7a422eec3d6c5b71b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:58.408620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:58.468104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1755247ff27642a7a422eec3d6c5b71b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:58.469066 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1755247ff27642a7a422eec3d6c5b71b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:58.469862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1755247ff27642a7a422eec3d6c5b71b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:58.486579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1755247ff27642a7a422eec3d6c5b71b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:58.503015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1755247ff27642a7a422eec3d6c5b71b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:58.525441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1180/2378] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:06:58 2022] DELETE /identity/v3/users/1755247ff27642a7a422eec3d6c5b71b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:06:58.530491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:58.531022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:58.531124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:58.551660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:06:58.552865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:58.622320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1199/2379] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:06:58.631483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:58.633043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:58.669110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:06:58.712420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd9878-b99d-4893-996a-6c01e6e66c47 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:58.716287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:58.716913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:58.717190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:58.717459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:58.731328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:58.732772 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20687191-9230-4c23-8488-51ce98e70b05 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:58.742903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1181/2380] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:06:59.030908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.032510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:59.045312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5738da-826c-4480-b10b-2ce76cf909bd None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.050778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.051522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.051818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.052204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] PATH_INFO: `/v3/projects/149c64dc002f4a49bcedd1e681e0bace` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.053351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149c64dc002f4a49bcedd1e681e0bace)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.055852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.105803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149c64dc002f4a49bcedd1e681e0bace', 'request_id': 'req-75c7bbc9-2272-4224-b505-fec74bfbc771'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:59.111518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149c64dc002f4a49bcedd1e681e0bace', 'request_id': 'req-75c7bbc9-2272-4224-b505-fec74bfbc771'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:59.117689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149c64dc002f4a49bcedd1e681e0bace', 'request_id': 'req-75c7bbc9-2272-4224-b505-fec74bfbc771'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:59.132883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] The token cache is being invalidate because project 149c64dc002f4a49bcedd1e681e0bace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:06:59.133181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:59.133843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-75c7bbc9-2272-4224-b505-fec74bfbc771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:06:59.135384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1200/2381] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:06:59 2022] DELETE /identity/v3/projects/149c64dc002f4a49bcedd1e681e0bace => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:06:59.140727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.141016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.141286 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.160871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:06:59.162119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:59.222750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1182/2382] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:06:59.230459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.230731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.230999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.250854 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:06:59.252054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:59.260179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1201/2383] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:06:59.264714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.276089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507edf7-6563-4b55-9cad-7cbdcb3e0e83 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.283622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.284570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.284995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.285308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.286426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:06:59.286923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.289457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.306204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1183/2384] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:06:59.312982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.326189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8eb9b-97f2-43de-a41d-ba4f3d9e764a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.334187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.335002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.335356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.335706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.336310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.338280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.344781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:06:59.370954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1202/2385] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:06:59 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:06:59.375163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.385458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008a737-7882-44a8-bfc9-1d7730c5a004 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.444508 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.445117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.445391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.445759 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.446403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.448325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.472526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1184/2386] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:06:59 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:06:59.478973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.496365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f449929-ddcb-4071-ab61-54f7d24bf367 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.505085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.505705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.505994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.506252 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.506898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.508531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.518801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1203/2387] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:06:59.522913 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.533995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64274fc8-87ec-47b4-ad32-569a388778d3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.539788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.540412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.540678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.540949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.541622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.543251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ba81-2d2d-40f1-882d-ad6e546b5a87 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.554915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1185/2388] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:06:59.559283 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.570300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5438-9b2b-49a3-853a-6bb890216038 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.575465 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.576124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.576404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.576678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] PATH_INFO: `/v3/projects/fd0e11ad6b3f4ac4a9d342484031f1c1/users/f3a3e076c5904520acefb22b17832d59/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.592362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0e11ad6b3f4ac4a9d342484031f1c1, user_id=f3a3e076c5904520acefb22b17832d59, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.595090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.610185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1204/2389] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:06:59 2022] PUT /identity/v3/projects/fd0e11ad6b3f4ac4a9d342484031f1c1/users/f3a3e076c5904520acefb22b17832d59/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:06:59.621501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.621960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.622370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.662450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] MFA Rules not processed for user `f3a3e076c5904520acefb22b17832d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:06:59.663698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:59.686823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:06:59.738643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1186/2390] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:06:59.749481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:06:59.751331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:06:59.809398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921910f-2ca8-4b13-a5b1-a2257b635601 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:06:59.813126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:06:59.813767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:06:59.814039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:06:59.814306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:06:59.822925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:06:59.824360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c319842-568e-488a-8cc0-a65cd4ae8a37 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:06:59.833805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1205/2391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:07:09.735511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=17 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:09.829244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:14.299534 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:07:14.365282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9e2ba-310c-4e98-a2ac-6102df29db0d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:07:14.370657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:07:14.371288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:14.371564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:14.371831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] PATH_INFO: `/v3/users/f3a3e076c5904520acefb22b17832d59` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:14.373464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a3e076c5904520acefb22b17832d59)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:07:14.374937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:07:14.427316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a3e076c5904520acefb22b17832d59'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:14.427978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a3e076c5904520acefb22b17832d59'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:14.428591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a3e076c5904520acefb22b17832d59'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:14.440678 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a3e076c5904520acefb22b17832d59'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:14.447548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a3e076c5904520acefb22b17832d59'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:14.462520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1187/2392] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:07:14 2022] DELETE /identity/v3/users/f3a3e076c5904520acefb22b17832d59 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:07:14.467235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:14.467636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:14.467916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:14.487731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:07:14.488965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:14.556306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1206/2393] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:07:14.565587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:07:14.567001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:14.630016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f708-557f-4d58-929e-93a686bb39be admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:07:14.633666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:07:14.634282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:14.634558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:14.634827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:14.643594 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:07:14.645024 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ed3eb-c20d-446f-b740-3d6070dd888b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:07:14.654370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1188/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:07:14.928628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:07:14.930421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:14.942958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8742719-acdb-46a9-a199-c74eceb0bea0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:07:14.948222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:07:14.948873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:14.949315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:14.949607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] PATH_INFO: `/v3/projects/fd0e11ad6b3f4ac4a9d342484031f1c1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:14.950746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0e11ad6b3f4ac4a9d342484031f1c1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:07:14.952653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:07:15.002241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0e11ad6b3f4ac4a9d342484031f1c1', 'request_id': 'req-854adf79-fe60-443a-b927-2877ae05fe79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:15.007572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0e11ad6b3f4ac4a9d342484031f1c1', 'request_id': 'req-854adf79-fe60-443a-b927-2877ae05fe79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:15.014685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0e11ad6b3f4ac4a9d342484031f1c1', 'request_id': 'req-854adf79-fe60-443a-b927-2877ae05fe79'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:15.031341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] The token cache is being invalidate because project fd0e11ad6b3f4ac4a9d342484031f1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:07:15.031622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:15.032271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:07:15.033822 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1207/2395] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:07:14 2022] DELETE /identity/v3/projects/fd0e11ad6b3f4ac4a9d342484031f1c1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:07:18.544416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:18.544929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:18.544929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:18.564507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] MFA Rules not processed for user `1fa6eb4bc1e54b6a80b038ea768d902c`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:07:18.565732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:18.582710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:18.621944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1189/2396] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:07:25.028058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:28.615315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=15 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:45.738290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:07:45.739855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:45.802589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854adf79-fe60-443a-b927-2877ae05fe79 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:07:45.808731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:07:45.809586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:45.809897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:45.810215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:45.813145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:45.889000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:07:45.890497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:07:45.901273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1208/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:07:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:07:47.091631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:07:47.102908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb727817-0abf-44b1-94ce-dac2575e1ad5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:07:47.107366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:07:47.107966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:07:47.108244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:07:47.108511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:07:47.110547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:07:47.131321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:07:47.133449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:07:47.147415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1190/2398] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:07:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:07:55.896634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=54 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:07:57.142602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:08:27.205384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:08:27.207590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:27.222740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c1186-cf2d-4989-a527-d844ebd5b870 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:08:27.229841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:08:27.230455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:08:27.230834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:08:27.231123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:08:27.233091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:27.292529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:08:27.294188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:08:27.303696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1209/2399] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:08:30.879516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:08:30.881115 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:30.948239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dbb44-fc18-45a8-a522-d82133a30e62 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:08:30.955561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:08:30.956570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:08:30.957027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:08:30.957448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] PATH_INFO: `/v3/users/1fa6eb4bc1e54b6a80b038ea768d902c` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:08:30.959063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa6eb4bc1e54b6a80b038ea768d902c)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:08:30.962149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:08:31.011008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa6eb4bc1e54b6a80b038ea768d902c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.011712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa6eb4bc1e54b6a80b038ea768d902c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.012356 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa6eb4bc1e54b6a80b038ea768d902c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.024537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa6eb4bc1e54b6a80b038ea768d902c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.031847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa6eb4bc1e54b6a80b038ea768d902c'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.045694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1191/2400] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:08:30 2022] DELETE /identity/v3/users/1fa6eb4bc1e54b6a80b038ea768d902c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:08:31.226344 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:08:31.242270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e404f-7c80-4991-a15a-939df3e71a5e service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:08:31.301747 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:08:31.302393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:08:31.302691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:08:31.302956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] PATH_INFO: `/v3/projects/efac6ddeea0d4d458b7de89c533e540d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:08:31.304223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=efac6ddeea0d4d458b7de89c533e540d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:08:31.306186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:08:31.363160 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efac6ddeea0d4d458b7de89c533e540d', 'request_id': 'req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.369430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efac6ddeea0d4d458b7de89c533e540d', 'request_id': 'req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.375783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efac6ddeea0d4d458b7de89c533e540d', 'request_id': 'req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.390915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] The token cache is being invalidate because project efac6ddeea0d4d458b7de89c533e540d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:08:31.391196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.391894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e9ce5df8-c1c6-4db4-99f9-36e0751ebd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:08:31.393469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1210/2401] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:08:31 2022] DELETE /identity/v3/projects/efac6ddeea0d4d458b7de89c533e540d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:08:35.471336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:08:35.474031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:35.532470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5e217-c389-4e0c-8bba-813a0abdad11 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:08:35.565756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:08:35.587039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:08:35.587657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:08:35.587997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:08:35.588274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:08:35.590194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:35.600557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:08:35.601727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:08:35.610880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1192/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:08:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:08:41.387832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:08:45.606766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=10 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:08:55.768351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:08:55.768835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:08:55.768835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:08:55.788654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] MFA Rules not processed for user `a5aece8a7f2342cfabd9a3f8707a4470`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:08:55.789891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:08:55.844494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1211/2403] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:08:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:09:02.171934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:02.173529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:02.257124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91672925-335b-4a7f-ab2f-07a315e2c4f1 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:02.262970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:02.263584 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:02.263867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:02.264137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:02.266329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:02.279315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:02.280532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:02.293204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1193/2404] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:02.497574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:02.499198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:02.515434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491bf6c-533a-4b48-9480-9aa25479e329 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:02.521082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:02.521735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:02.522074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:02.522400 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:02.524207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:02.592136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:02.593884 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:02.603195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1212/2405] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:07.176906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:07.188688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154ef2f-6784-4977-9ba9-f2cdf61b1839 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:07.195565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:07.196161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:07.196433 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:07.196705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:07.198910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:07.265801 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:07.267424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9db91-9860-4250-9d32-bfebff3422a6 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:07.277189 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1194/2406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:10.633892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:10.647234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3d8bf-25c3-4bbc-988d-c2ef0f975f20 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:10.653956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:10.654912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:10.655390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:10.655858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:10.659191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:10.672973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:10.674756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:10.686762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1213/2407] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:17.272490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=62 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:09:20.682412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=59 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:09:21.283968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:21.284526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:21.284526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:21.314520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] MFA Rules not processed for user `454c2bf24ed94f82b453df36e6e8eac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:09:21.316276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:21.324423 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1195/2408] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:09:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:09:21.641350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:21.642886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:21.655036 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f23be2-b8a7-4ba6-ab4f-815df589c1be service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:21.659403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:21.659999 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:21.660275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:21.660540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:21.662556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:21.677319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:21.678784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:21.686974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1214/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:24.272295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:24.273970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:24.354134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf61fb-fadf-4490-9762-b9216d8feadd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:24.359325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:24.359943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:24.360221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:24.360490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:24.362266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:24.374403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:24.375991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:24.385857 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1196/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:31.683587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:09:34.381919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=35 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:09:35.943521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:09:35.957593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c31c-51f4-4072-aa52-a5315059056c service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:09:35.963132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:09:35.963896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:09:35.964208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:09:35.964485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:09:35.966342 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:09:35.979518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:09:35.981140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:09:35.992048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1215/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:09:45.987795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:10:22.384201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:22.394926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ebf39-8fb2-421f-8025-d3329143a4d5 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:22.399294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:22.399885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:22.400157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:22.400424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:22.402540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:22.477574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:22.479019 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:22.489806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1197/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:10:25.593229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:25.605600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a7ba-e15c-45b4-b197-d317cab6aa0f service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:25.610984 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:25.611724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:25.612183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:25.612518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:25.615080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:25.633427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:25.639952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:25.654476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1216/2413] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:10:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:10:25.664942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:25.666491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:25.683301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427eb38-0643-40a1-89d7-9f2c1e468098 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:25.688895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:25.689719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:25.690179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:25.690450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] PATH_INFO: `/v3/users/a5aece8a7f2342cfabd9a3f8707a4470` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:25.691660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5aece8a7f2342cfabd9a3f8707a4470)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:25.695881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:25.750783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5aece8a7f2342cfabd9a3f8707a4470'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:25.751442 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5aece8a7f2342cfabd9a3f8707a4470'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:25.752046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5aece8a7f2342cfabd9a3f8707a4470'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:25.764059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5aece8a7f2342cfabd9a3f8707a4470'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:25.771209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-92231908-e88e-470b-98c4-a78d05c4f115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5aece8a7f2342cfabd9a3f8707a4470'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:25.786110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1198/2414] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:10:25 2022] DELETE /identity/v3/users/a5aece8a7f2342cfabd9a3f8707a4470 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:10:25.965327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:25.976270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802f6c7-35fa-45bf-9840-a92c52cc9dba service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.035504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.036128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.036401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.036668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] PATH_INFO: `/v3/projects/db932807b4634440ba61a82446e82874` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.037879 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=db932807b4634440ba61a82446e82874)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.039816 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.090111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db932807b4634440ba61a82446e82874', 'request_id': 'req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:26.095676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db932807b4634440ba61a82446e82874', 'request_id': 'req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:26.101913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db932807b4634440ba61a82446e82874', 'request_id': 'req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:26.117186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] The token cache is being invalidate because project db932807b4634440ba61a82446e82874 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:10:26.117473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:26.118229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ecaa5cfb-5a7d-4719-834c-0a820e6c3fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:10:26.119894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1217/2415] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:10:25 2022] DELETE /identity/v3/projects/db932807b4634440ba61a82446e82874 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:10:26.126070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.126359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.126634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.146558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:26.147698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:26.210347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1199/2416] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:26.218298 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.218571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.218837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.240029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:26.241239 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:26.250558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1218/2417] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:26.254945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.265144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38dae-3c30-4b5f-bad6-f87d0d3e34cd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.270922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.271525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.271803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.272036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.272714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:10:26.273012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.274653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.276108 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:10:26.287628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1200/2418] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:10:26.292222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.302928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60442a-9a2c-430c-99db-bd5585ffb626 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.310638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.311522 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.311891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.312246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.313206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.316334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.350391 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1219/2419] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:10:26 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:10:26.354771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.365177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a6787-10ea-464b-b7d6-5f8073437842 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.422538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.423064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.423343 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.423618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.424259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.426214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.452711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1201/2420] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:10:26 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:10:26.453977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.464959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc2ea8-6f2f-4569-b44d-5be65d2358d3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.470529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.471125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.471393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.471651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.472287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.473939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.484505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1220/2421] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:10:26.488970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.499721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280df62-447f-4e46-9a10-9e44f25b9d46 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.510303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.510976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.511248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.511514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.512165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.513807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb379dbf-fa56-4a7f-9926-fdd421fd19b9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.525373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1202/2422] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:10:26.530145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.540937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938bee-314e-4f7a-ba3b-7d5016d634e3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.552765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.553449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.553757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.554027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] PATH_INFO: `/v3/projects/129d0816f8ef4b4f8c8d626b49a03679/users/31a49b84974b480985d0b002fc740e95/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.570629 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129d0816f8ef4b4f8c8d626b49a03679, user_id=31a49b84974b480985d0b002fc740e95, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.573388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.590655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1221/2423] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:10:26 2022] PUT /identity/v3/projects/129d0816f8ef4b4f8c8d626b49a03679/users/31a49b84974b480985d0b002fc740e95/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:10:26.596001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714639-4f1f-4e7f-81ca-e53b6570e74d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.596281 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714639-4f1f-4e7f-81ca-e53b6570e74d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.596551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714639-4f1f-4e7f-81ca-e53b6570e74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.615473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f5714639-4f1f-4e7f-81ca-e53b6570e74d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:26.616669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f5714639-4f1f-4e7f-81ca-e53b6570e74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:26.678826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1203/2424] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:26.689356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:26.692372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:26.714943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:10:26.761470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115c01-ca18-474a-8d70-a4d681238678 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:26.765481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:26.766142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:26.766422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:26.766649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:26.776673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:26.778182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:26.787893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1222/2425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:10:30.964131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a3a-49f8-42c9-8164-ac0e085c6796 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:30.964729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a3a-49f8-42c9-8164-ac0e085c6796 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:30.964729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75a3a-49f8-42c9-8164-ac0e085c6796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:31.007301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f9b75a3a-49f8-42c9-8164-ac0e085c6796 None None] MFA Rules not processed for user `31a49b84974b480985d0b002fc740e95`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:31.007301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f9b75a3a-49f8-42c9-8164-ac0e085c6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:31.095629 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1204/2426] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:31.111765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:10:31.113270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:31.208244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79627b8-acb3-47ad-9431-a6c1f0e79a22 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:10:31.214645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:10:31.215761 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:31.216214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:31.216482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:31.225059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:10:31.227043 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293a5f3-849d-4c25-afd7-1213f9811004 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:10:31.237460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1223/2427] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:10:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:10:32.724547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c89f4a-5820-42e1-a8ca-e7131827dc98 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:32.725052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c89f4a-5820-42e1-a8ca-e7131827dc98 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:32.725169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c89f4a-5820-42e1-a8ca-e7131827dc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:32.732688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:10:32.745617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-d1c89f4a-5820-42e1-a8ca-e7131827dc98 None None] MFA Rules not processed for user `31a49b84974b480985d0b002fc740e95`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:32.747230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d1c89f4a-5820-42e1-a8ca-e7131827dc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:32.755453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1205/2428] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:37.770425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:10:37.770986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:10:37.770986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:10:37.790401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] MFA Rules not processed for user `3a40f55377b64f6f93784eefaf28152e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:10:37.791846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:10:37.849130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1224/2429] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:10:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:10:42.753211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:10:47.842518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=79 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:11:13.169348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:11:13.169942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:11:13.169942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:11:13.200048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] MFA Rules not processed for user `454c2bf24ed94f82b453df36e6e8eac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:11:13.201248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:11:13.277107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1206/2430] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:11:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:11:13.608152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:11:13.609728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:11:13.675260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe392dc-88f0-440a-b0f4-3d598cd77c50 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:11:13.680020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:11:13.680623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:11:13.680909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:11:13.681171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:11:13.683199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:11:13.694019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:11:13.695539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c913be5-f1ef-47b6-8db6-5b3b3191d845 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:11:13.703582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1225/2431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:11:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:11:23.275124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=37 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:11:23.700324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=28 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:03.276848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:03.278455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:03.389958 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e36f6-03cd-4043-af69-dba8095f39d8 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:03.396375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:03.396988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:03.397269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:03.397569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:03.399347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:03.411145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:03.412718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:03.422699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1207/2432] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:13.418856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=44 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:29.570947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:29.572174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:29.572296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:29.606058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:29.608042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:29.674348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1226/2433] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:12:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:29.732992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:29.734529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:29.805927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830043e-614b-4238-a825-cfd71b108b1f service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:29.811427 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:29.812107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:29.812463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:29.812939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:29.822180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:29.823767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:29.831872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1208/2434] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:12:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:12:39.582768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:39.594446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed0a9b-58f5-4510-a8cb-d48f20455084 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:39.599126 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:39.599741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:39.600014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:39.600282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:39.602318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:39.638474 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:39.639963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:39.660489 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1227/2435] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:39.828705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:43.380554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:43.383188 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:43.399241 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d6fb7-b919-49c0-baf3-40bee1577763 service swift] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:43.407169 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:43.407912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:43.408260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:43.408615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:43.410525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:43.423617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:43.425432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:43.438095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1209/2436] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:43.735630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:43.737636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:43.749925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de7245-f3c8-425a-a70b-c3564f53eca1 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:43.754595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:43.755209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:43.755482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:43.755746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] PATH_INFO: `/v3/users/3a40f55377b64f6f93784eefaf28152e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:43.756857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a40f55377b64f6f93784eefaf28152e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:43.759121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:43.812890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a40f55377b64f6f93784eefaf28152e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:43.820260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a40f55377b64f6f93784eefaf28152e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:43.832800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a40f55377b64f6f93784eefaf28152e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:43.842205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a40f55377b64f6f93784eefaf28152e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:43.842870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-3f51518c-46e9-485d-afe4-3c3fa17b3a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a40f55377b64f6f93784eefaf28152e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:43.848960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1228/2437] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:12:43 2022] DELETE /identity/v3/users/3a40f55377b64f6f93784eefaf28152e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:44.026387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.040816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fff1c-ad46-4f3f-9524-3b0a076fe5c3 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.103563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.104201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.104475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.104749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] PATH_INFO: `/v3/projects/da5173bee1cf4ec09240370892c5b9e3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.115384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5173bee1cf4ec09240370892c5b9e3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.117364 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.165472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5173bee1cf4ec09240370892c5b9e3', 'request_id': 'req-1a457395-b5d0-44b9-a5da-79ce6e8498ff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:44.171436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5173bee1cf4ec09240370892c5b9e3', 'request_id': 'req-1a457395-b5d0-44b9-a5da-79ce6e8498ff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:44.180712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5173bee1cf4ec09240370892c5b9e3', 'request_id': 'req-1a457395-b5d0-44b9-a5da-79ce6e8498ff'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:44.193919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] The token cache is being invalidate because project da5173bee1cf4ec09240370892c5b9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:12:44.194260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:44.194851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-1a457395-b5d0-44b9-a5da-79ce6e8498ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:44.196367 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1210/2438] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:12:44 2022] DELETE /identity/v3/projects/da5173bee1cf4ec09240370892c5b9e3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:44.201311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.201639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.202130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.230090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:44.265307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:44.266585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.331813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1229/2439] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:12:44.339898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.340180 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.340477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.360827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:44.362068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.370430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1211/2440] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:44.374717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.385635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c85e6-7a6b-4de1-a9b8-89dec0f10ef3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.390433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.391047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.391318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.391586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.392254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:12:44.392533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.394183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.406918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1230/2441] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:12:44.411279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.421329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ff0f5-bd7b-4daa-aabe-497015ce8b62 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.426657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.427502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.427783 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.428149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.429378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.431954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c54537-2aff-4f39-a86b-8e857f12a431 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.475414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1212/2442] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:12:44.479858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.491002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc7ce-16f5-46fc-b986-bb4e1e5c5a3f None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.491289 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc7ce-16f5-46fc-b986-bb4e1e5c5a3f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.491757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fc7ce-16f5-46fc-b986-bb4e1e5c5a3f None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.492432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd36d7d-cc31-46ec-9d1c-a98d917ab872 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.493466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1213/2443] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:12:44.499291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.499582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.499905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.557406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:44.558618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.559312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.559493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.559774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.560388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.561173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:44.562442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.562967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.589843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1231/2444] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:12:44.594142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.606083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f415cb-d313-453a-9e4b-c0b0cdc78216 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.611504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.611938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.612413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.612730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.614125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.615711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.627156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1232/2445] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:12:44.631244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.643373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da7e6-a180-4f7b-ab8f-d7b6f81580f2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.647973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.648612 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.648888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.649153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.649876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.651475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.658385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1214/2446] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:44.661929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1233/2447] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:12:44.666005 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.672851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.674704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.676154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6973c06-8ff9-45c9-bd45-aaec217594bd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.681951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.682656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.682935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.683207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] PATH_INFO: `/v3/projects/3034511ba1cc438498ececa7de891f50/users/1fdb83232f6542ce8c39014575792177/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.697911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3034511ba1cc438498ececa7de891f50, user_id=1fdb83232f6542ce8c39014575792177, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.700621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7cbf9-5c39-4ace-8d4e-03b6ddf339f2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.715265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1215/2448] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:44 2022] PUT /identity/v3/projects/3034511ba1cc438498ececa7de891f50/users/1fdb83232f6542ce8c39014575792177/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:44.726641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.727035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.727314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.727845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:44.738837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0748d-4259-4aee-b6c4-6779bc082726 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.755934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] MFA Rules not processed for user `1fdb83232f6542ce8c39014575792177`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:44.757155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.787341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.788016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.788292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.788562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.822058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1216/2449] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:12:44.833929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:44.835430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:44.864833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.866340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913d28c-f394-476f-962b-8fb95f10c85b service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.877393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1234/2450] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:12:44.889165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:44.910518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72a7b4-b22d-4eda-bd92-7cdc37e94693 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:44.916220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:44.916842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.917117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.917389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.925633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:44.927208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff8013-8b6d-478e-a897-b56cf3afa90d service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:44.936902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1217/2451] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:12:44.949118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.949408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.949727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.951098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1235/2452] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:12:44.955732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650f57a-90e7-4cc1-9afb-e2c944cefee7 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:44.956243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650f57a-90e7-4cc1-9afb-e2c944cefee7 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:44.956513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650f57a-90e7-4cc1-9afb-e2c944cefee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:44.976924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-a650f57a-90e7-4cc1-9afb-e2c944cefee7 None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:44.978176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a650f57a-90e7-4cc1-9afb-e2c944cefee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:45.031941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1218/2453] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:12:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:45.044347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:45.046148 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:45.060117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e6a5c-3a63-48e2-807e-548a64db31a1 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:45.064174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:45.064831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:45.065169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:45.065681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:45.074935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:45.076493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78484e52-8bcf-481d-a3b5-c15b157e9fdc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:45.085325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1236/2454] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:45.173932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859dce-a65e-47ad-a745-c5593eda8a19 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:45.174272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859dce-a65e-47ad-a745-c5593eda8a19 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:45.174548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2859dce-a65e-47ad-a745-c5593eda8a19 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:45.175969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1219/2455] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:12:45.183493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:45.183833 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:45.184175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:45.205724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:45.207274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:45.217343 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1237/2456] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:12:45.230279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2859dce-a65e-47ad-a745-c5593eda8a19 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:45.243495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2859dce-a65e-47ad-a745-c5593eda8a19 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:45.247719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:45.248533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:45.248878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:45.249237 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:45.264162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:45.266071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:45.278295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1220/2457] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:46.351704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:46.353477 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:46.438356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcbfaf6-cbe6-4165-8875-5eb97e47e6aa None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:46.443078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:46.443782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:46.444055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:46.444320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:46.446525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:46.461223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:46.462745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334226e-9677-42b8-8162-b1dcadacebda service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:46.474772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1238/2458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:46.940814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:46.950201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061da2d-10a8-48e0-8257-4cb1a5379038 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:46.954575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:46.955170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:46.955441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:46.955706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:46.957677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:46.971688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:46.973082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:46.984425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1221/2459] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:51.328501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:51.328986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:51.329096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:51.338183 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:51.349741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:51.350966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:51.407850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1239/2460] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:12:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:51.420558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:51.422202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:51.496273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcd88d-ccf8-44d8-bad9-b3eada0323eb service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:51.502383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:51.502997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:51.503276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:51.503551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:51.514248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:51.515806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b54dc-ce51-4bbd-9495-8fa1be6e829d service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:51.524247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1222/2461] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:12:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:12:54.540776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:54.619785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7d8c4-c360-462a-8432-0e6bf499f091 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:54.624585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:54.625222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:54.625494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:54.625787 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] PATH_INFO: `/v3/users/1fdb83232f6542ce8c39014575792177` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:54.626941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdb83232f6542ce8c39014575792177)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:54.628923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:54.690202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdb83232f6542ce8c39014575792177'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:54.697137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdb83232f6542ce8c39014575792177'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:54.709398 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdb83232f6542ce8c39014575792177'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:54.718791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdb83232f6542ce8c39014575792177'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:54.719427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdb83232f6542ce8c39014575792177'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:54.725979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1240/2462] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:12:54 2022] DELETE /identity/v3/users/1fdb83232f6542ce8c39014575792177 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:54.730839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427af124-d87a-401c-a784-88cf69159f13 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:54.731358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427af124-d87a-401c-a784-88cf69159f13 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:54.731358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427af124-d87a-401c-a784-88cf69159f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:54.752587 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-427af124-d87a-401c-a784-88cf69159f13 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:54.753851 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-427af124-d87a-401c-a784-88cf69159f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:54.803741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:54.818355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1223/2463] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:12:54.828288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:54.829757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:54.867024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:54.901202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd33be0-aeae-4481-9321-254ac8de738f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:54.904708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:54.905313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:54.905620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:54.905890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:54.916073 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:54.917634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a84445-f987-4507-bdfe-af07f8d3afb2 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:54.927901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1241/2464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:55.221009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427af124-d87a-401c-a784-88cf69159f13 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.222669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-427af124-d87a-401c-a784-88cf69159f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:55.254751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427af124-d87a-401c-a784-88cf69159f13 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.259827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.261323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.261982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.262265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] PATH_INFO: `/v3/projects/3034511ba1cc438498ececa7de891f50` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.263448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3034511ba1cc438498ececa7de891f50)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.265363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.322924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3034511ba1cc438498ececa7de891f50', 'request_id': 'req-8343d447-8133-4035-8673-68be6de41f33'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:55.328900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3034511ba1cc438498ececa7de891f50', 'request_id': 'req-8343d447-8133-4035-8673-68be6de41f33'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:55.343375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3034511ba1cc438498ececa7de891f50', 'request_id': 'req-8343d447-8133-4035-8673-68be6de41f33'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:55.355308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] The token cache is being invalidate because project 3034511ba1cc438498ececa7de891f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:12:55.355609 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:55.356422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8343d447-8133-4035-8673-68be6de41f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:55.358329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1224/2465] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:12:55 2022] DELETE /identity/v3/projects/3034511ba1cc438498ececa7de891f50 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:55.363371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.363697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.363971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.386199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:55.387430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:55.462227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1242/2466] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:55.471183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.471539 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.471748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.494879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:55.496595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:55.513768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1225/2467] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:55.518169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.529835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff4b8c-72fc-42a4-8d02-e57c13939b54 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.535367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.536009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.536262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.536598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.537440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:12:55.537779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.542723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.558838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1243/2468] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:12:55.563300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.574923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d0998-13d4-45f5-8ac8-c4e30d21d8f2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.579862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.580520 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.580802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.581070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.581811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.583799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.621711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1226/2469] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:55.627586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.664579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203ed55-39d5-4d76-994f-40efc5179456 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.725181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.725736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.726010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.726274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.726994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.728991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.754024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1244/2470] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:55.758622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.769437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcbe7f-b4cc-4ab7-8e13-c7f70a57b9dc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.774013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.774612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.774888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.775172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.775889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.777780 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.788814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1227/2471] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:55.793027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.804213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2562-0b2e-487e-bd99-beac92567fca admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.810411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.811088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.811387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.811655 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.812323 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.814273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c639a8c-bab5-4400-9b5f-3351d6a50e09 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.826420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1245/2472] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:55.832450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:55.843447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e2606-2064-48a8-9437-69fc88704530 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:55.848420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:55.849098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.849369 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.849674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] PATH_INFO: `/v3/projects/9444da9406024e0c9fff9b36f6b90442/users/5727bbea6ab8426aa1b02821acd021a5/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.865480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9444da9406024e0c9fff9b36f6b90442, user_id=5727bbea6ab8426aa1b02821acd021a5, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:55.868313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:55.884513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1228/2473] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:55 2022] PUT /identity/v3/projects/9444da9406024e0c9fff9b36f6b90442/users/5727bbea6ab8426aa1b02821acd021a5/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:55.895115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:55.895444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:55.895791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:55.924090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] MFA Rules not processed for user `5727bbea6ab8426aa1b02821acd021a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:55.925759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:55.955002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:55.999548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1246/2474] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:56.000843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.072079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715225d3-28a8-4387-8f4e-c54cd1a148a6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.076911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.077529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.077834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.078106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.078744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.080667 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.104542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:56.112820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1229/2475] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:12:55 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:56.118366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.130613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c76d70-329a-4ffb-9c21-2a2d6e5ff775 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.190805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.191428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.191704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.191974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.192621 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.194678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.219804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1247/2476] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:12:56 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:56.224251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.235334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354bf0f-ec53-4e27-a944-91586a9fa34c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.240209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.240809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.241107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.241383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.242087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.243847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.255785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1230/2477] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:12:56.260315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.275827 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7d039-71ed-483b-bc35-ce11a4e33416 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.281003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.281802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.282143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.282510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] PATH_INFO: `/v3/projects/877e113fe17345a3b4751df4ea2b0d7d/users/e2fdbb09adb74b20a578bc42de0fe095/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.298232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=877e113fe17345a3b4751df4ea2b0d7d, user_id=e2fdbb09adb74b20a578bc42de0fe095, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.301723 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.316805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1248/2478] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:56 2022] PUT /identity/v3/projects/877e113fe17345a3b4751df4ea2b0d7d/users/e2fdbb09adb74b20a578bc42de0fe095/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:56.321034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.332916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baa255-b7b0-4453-b3c0-a599d2bce33a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.393884 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.394632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.394981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.395328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.396039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.397766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0bb4-66b5-4961-8d73-1416ba64e085 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.411344 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1231/2479] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:56.414406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.425232 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9484a-9f75-46e2-b2b4-7a759de524fa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.430292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.430963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.431242 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.431504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] PATH_INFO: `/v3/projects/877e113fe17345a3b4751df4ea2b0d7d/users/e2fdbb09adb74b20a578bc42de0fe095/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.433444 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=877e113fe17345a3b4751df4ea2b0d7d, user_id=e2fdbb09adb74b20a578bc42de0fe095, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.436284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.447418 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1249/2480] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:56 2022] PUT /identity/v3/projects/877e113fe17345a3b4751df4ea2b0d7d/users/e2fdbb09adb74b20a578bc42de0fe095/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:56.459391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.459695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.459968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.489568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] MFA Rules not processed for user `e2fdbb09adb74b20a578bc42de0fe095`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:56.490960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:56.577689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1232/2481] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:56.591509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.592361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:56.694955 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6f58e-13ab-414d-a5fb-26e545c9ddcc admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.700213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.700840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.701256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.701531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.715819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.717288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d204a-afdc-4a52-a25f-9af153f6135b service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.726896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1250/2482] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:56.757310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:56.825606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a52654-30e9-45b1-9eeb-9f49188fefe5 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:56.831920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:56.832710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:56.832994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:56.833275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] PATH_INFO: `/v3/users/5727bbea6ab8426aa1b02821acd021a5` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:56.834656 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5727bbea6ab8426aa1b02821acd021a5)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:56.836704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:56.848572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:56.925826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5727bbea6ab8426aa1b02821acd021a5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:56.926687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5727bbea6ab8426aa1b02821acd021a5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:56.927319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5727bbea6ab8426aa1b02821acd021a5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:56.967384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5727bbea6ab8426aa1b02821acd021a5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:56.984813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5727bbea6ab8426aa1b02821acd021a5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.000710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1233/2483] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:12:56 2022] DELETE /identity/v3/users/5727bbea6ab8426aa1b02821acd021a5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:57.005455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:57.005781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:57.006051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:57.026380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:57.027613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:57.046366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:57.092037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1251/2484] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:57.102167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:57.103612 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:57.170395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a6459-5a60-440a-9912-61b05b26a533 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:57.174087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:57.174710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:57.174989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:57.175263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:57.186536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:57.188151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:57.199053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1234/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:57.483899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:57.485757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:57.498896 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12940da-91f1-418b-a464-b6a6ddf59f2c None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:57.505607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:57.506321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:57.506632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:57.507120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] PATH_INFO: `/v3/projects/9444da9406024e0c9fff9b36f6b90442` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:57.508492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9444da9406024e0c9fff9b36f6b90442)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:57.511156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:57.564726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9444da9406024e0c9fff9b36f6b90442', 'request_id': 'req-a31a08fe-b803-4319-9394-7cf6e4678db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.570959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9444da9406024e0c9fff9b36f6b90442', 'request_id': 'req-a31a08fe-b803-4319-9394-7cf6e4678db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.579338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9444da9406024e0c9fff9b36f6b90442', 'request_id': 'req-a31a08fe-b803-4319-9394-7cf6e4678db0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.594824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] The token cache is being invalidate because project 9444da9406024e0c9fff9b36f6b90442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:12:57.595497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.600732 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.602588 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1252/2486] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:12:57 2022] DELETE /identity/v3/projects/9444da9406024e0c9fff9b36f6b90442 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:57.606879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:57.617571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ba338-7a6f-4dfd-b6e1-073545cfdde0 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:57.678010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:57.678996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:57.679277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:57.679684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] PATH_INFO: `/v3/users/e2fdbb09adb74b20a578bc42de0fe095` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:57.680889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2fdbb09adb74b20a578bc42de0fe095)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:57.682901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:57.724179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:57.740840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2fdbb09adb74b20a578bc42de0fe095'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.741808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2fdbb09adb74b20a578bc42de0fe095'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.742410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2fdbb09adb74b20a578bc42de0fe095'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.759886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2fdbb09adb74b20a578bc42de0fe095'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.763961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-44703b9a-20b3-4209-aa76-df71e15673e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2fdbb09adb74b20a578bc42de0fe095'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:57.792815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1235/2487] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:12:57 2022] DELETE /identity/v3/users/e2fdbb09adb74b20a578bc42de0fe095 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:58.122032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.123510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:58.149074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a08fe-b803-4319-9394-7cf6e4678db0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.213378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.214095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.214366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.214630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] PATH_INFO: `/v3/projects/877e113fe17345a3b4751df4ea2b0d7d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.215743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=877e113fe17345a3b4751df4ea2b0d7d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.217694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.267181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:58.268840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877e113fe17345a3b4751df4ea2b0d7d', 'request_id': 'req-cdab6e66-c086-431c-b04d-b6b9bb48a2da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:58.277359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877e113fe17345a3b4751df4ea2b0d7d', 'request_id': 'req-cdab6e66-c086-431c-b04d-b6b9bb48a2da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:58.284893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877e113fe17345a3b4751df4ea2b0d7d', 'request_id': 'req-cdab6e66-c086-431c-b04d-b6b9bb48a2da'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:58.302963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] The token cache is being invalidate because project 877e113fe17345a3b4751df4ea2b0d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:12:58.303237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:58.304393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-cdab6e66-c086-431c-b04d-b6b9bb48a2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:12:58.306409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1253/2488] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:12:58 2022] DELETE /identity/v3/projects/877e113fe17345a3b4751df4ea2b0d7d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:58.311377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.311923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.312247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.332238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:58.333438 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:58.409295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1236/2489] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:58.411397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.411724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.412066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.433241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:58.434531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:58.446320 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1254/2490] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:58.447886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.458199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb0110-f1a9-4770-918e-79c93425fe35 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.462941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.463602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.463878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.464144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.465165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:12:58.465628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.467711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.485151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1237/2491] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:12:58.486353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.497038 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4846bb9-ccbf-4fa2-9886-2bda39031849 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.501806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.502413 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.502685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.503077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.503706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.505751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.543528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1255/2492] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:58.546064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.557566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffcc38-8e80-4aea-9214-e43e08468f56 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.617022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.617649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.617927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.618274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.618808 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.620796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.644223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1238/2493] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:58.648446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.659056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540224c-b531-4b53-b148-0a56a9733e70 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.664169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.664717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.664990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.665254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.665977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.667663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.679016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1256/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:58.683670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.694445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6670949-0fa3-40d5-8a8c-687ec8787a93 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.699748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.700675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.701125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.701628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.702285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.703951 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b51ea2-50a4-45d0-81dd-685f5ae2b68b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.718697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1239/2495] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:12:58.720029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.731212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb360-6015-4244-8533-09478c94505c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.736186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.736964 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.737248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.737518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] PATH_INFO: `/v3/projects/bf57457a9d8a4285b9812b50cc04f693/users/34ce119272af4243897b973a1cf415c3/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.753350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf57457a9d8a4285b9812b50cc04f693, user_id=34ce119272af4243897b973a1cf415c3, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.756056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:58.773228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1257/2496] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:58 2022] PUT /identity/v3/projects/bf57457a9d8a4285b9812b50cc04f693/users/34ce119272af4243897b973a1cf415c3/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:58.785010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.785272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.785636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.818187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] MFA Rules not processed for user `34ce119272af4243897b973a1cf415c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:58.819432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:58.823628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:58.890596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1240/2497] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:58.895025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:58.978271 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc9d5-aebe-46dd-8714-a60345ae03ee admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:58.983109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:58.983725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:58.984022 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:58.984301 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:58.985294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:58.987293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.029916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1258/2498] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:12:58 2022] POST /identity/v3/projects => generated 409 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:12:59.034535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.046328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e80bab-9ef4-4007-bb22-7945303033b9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.107497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.108330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.108840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.109332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.110219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.112485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.137676 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1241/2499] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:12:59 2022] POST /identity/v3/users => generated 311 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:12:59.142693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.153181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586683b0-4237-4d2c-9c1d-2722fff00bca admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.158173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.159016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.159526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.159979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.160979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.167476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.179569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1259/2500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:59.183909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.194302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e220d-d6cc-4157-977a-4013209807e5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.199030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.199704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.199981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.200251 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] PATH_INFO: `/v3/projects/f52d0d7793854837a134c7e9c54b690e/users/083589fa03af4ab9ab25f67341964234/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.219579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52d0d7793854837a134c7e9c54b690e, user_id=083589fa03af4ab9ab25f67341964234, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.223170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.239068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1242/2501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:59 2022] PUT /identity/v3/projects/f52d0d7793854837a134c7e9c54b690e/users/083589fa03af4ab9ab25f67341964234/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:59.243141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.253969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d47be-2829-4a00-a0a5-66ce7006487e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.304335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:59.316356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.317056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.317571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.317995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.318750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.320760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f75d078-13b9-49a7-bb43-e6dc713c2af4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.333626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1260/2502] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:12:59.338027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.350023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4562e-9a06-464b-a66d-615ba3fb8334 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.355260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.356012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.356274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.356576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] PATH_INFO: `/v3/projects/f52d0d7793854837a134c7e9c54b690e/users/083589fa03af4ab9ab25f67341964234/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.358542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52d0d7793854837a134c7e9c54b690e, user_id=083589fa03af4ab9ab25f67341964234, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.361222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.372523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1243/2503] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:12:59 2022] PUT /identity/v3/projects/f52d0d7793854837a134c7e9c54b690e/users/083589fa03af4ab9ab25f67341964234/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:12:59.383455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.383733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.383998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.413797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] MFA Rules not processed for user `083589fa03af4ab9ab25f67341964234`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:12:59.415618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:59.495960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1261/2504] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:12:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:12:59.503406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.505386 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:12:59.570878 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17731c3-489e-4d3f-9283-81913b3c1d3d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.574917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.575548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.575824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.576120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.586109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.587544 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.593480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:12:59.603516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1244/2505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:12:59.761082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:12:59.771094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a84933-40a6-4ae2-afa0-7e440b4ccb42 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:12:59.774680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:12:59.775290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:12:59.775566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:12:59.775834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:12:59.837468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:12:59.839109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249e35a-1b30-441a-9491-82722d2abf80 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:12:59.850371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1262/2506] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:12:59 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:09.845846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=70 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:10.407795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:10.481648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3f714-562d-4ce8-908a-94aa974ee7ef service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:10.486986 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:10.487653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:10.487926 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:10.488194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] PATH_INFO: `/v3/users/34ce119272af4243897b973a1cf415c3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:10.489422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ce119272af4243897b973a1cf415c3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:10.491383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:10.550317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ce119272af4243897b973a1cf415c3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:10.551012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ce119272af4243897b973a1cf415c3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:10.551617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ce119272af4243897b973a1cf415c3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:10.564413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ce119272af4243897b973a1cf415c3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:10.572436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ce119272af4243897b973a1cf415c3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:10.585741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1245/2507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:10 2022] DELETE /identity/v3/users/34ce119272af4243897b973a1cf415c3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:10.590591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:10.590901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:10.591177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:10.610537 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:10.611750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:10.672276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1263/2508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:10.684246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:10.685794 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:10.743716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92441e-4a42-4670-860f-d2f12df08763 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:10.747693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:10.748351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:10.748624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:10.748889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:10.758013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:10.759439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:10.768760 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1246/2509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:11.043666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:11.045406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:11.058664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d00d3-bf74-4380-ae54-438e2d4460de None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:11.064591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:11.065226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.065504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.065804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] PATH_INFO: `/v3/projects/bf57457a9d8a4285b9812b50cc04f693` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.067197 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf57457a9d8a4285b9812b50cc04f693)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:11.069128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:11.122717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf57457a9d8a4285b9812b50cc04f693', 'request_id': 'req-87d4f443-eec0-4b61-a774-261f476fb011'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.128941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf57457a9d8a4285b9812b50cc04f693', 'request_id': 'req-87d4f443-eec0-4b61-a774-261f476fb011'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.135374 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf57457a9d8a4285b9812b50cc04f693', 'request_id': 'req-87d4f443-eec0-4b61-a774-261f476fb011'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.148981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] The token cache is being invalidate because project bf57457a9d8a4285b9812b50cc04f693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:11.149261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.149899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.151431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1264/2510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:11 2022] DELETE /identity/v3/projects/bf57457a9d8a4285b9812b50cc04f693 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:11.155471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:11.165453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd247fc-b98a-4dd4-af30-78bc1fa9e83b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:11.184583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:11.229298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:11.229949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.230230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.230504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] PATH_INFO: `/v3/users/083589fa03af4ab9ab25f67341964234` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.231647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083589fa03af4ab9ab25f67341964234)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:11.233590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:11.283479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083589fa03af4ab9ab25f67341964234'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.284274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083589fa03af4ab9ab25f67341964234'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.284890 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083589fa03af4ab9ab25f67341964234'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.297101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083589fa03af4ab9ab25f67341964234'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.305093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-95699a47-0d97-4ba3-a3a2-8f52419bc200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083589fa03af4ab9ab25f67341964234'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.320503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1247/2511] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:11 2022] DELETE /identity/v3/users/083589fa03af4ab9ab25f67341964234 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:11.605858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:11.607309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:11.620716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4f443-eec0-4b61-a774-261f476fb011 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:11.689229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:11.689904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.690178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.690530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] PATH_INFO: `/v3/projects/f52d0d7793854837a134c7e9c54b690e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.691694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52d0d7793854837a134c7e9c54b690e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:11.693650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:11.727944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:11.748839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52d0d7793854837a134c7e9c54b690e', 'request_id': 'req-7acefc9e-b5c0-4c59-9777-86839ca484e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.754988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52d0d7793854837a134c7e9c54b690e', 'request_id': 'req-7acefc9e-b5c0-4c59-9777-86839ca484e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.761819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52d0d7793854837a134c7e9c54b690e', 'request_id': 'req-7acefc9e-b5c0-4c59-9777-86839ca484e9'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.779314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] The token cache is being invalidate because project f52d0d7793854837a134c7e9c54b690e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:11.779598 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.780238 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7acefc9e-b5c0-4c59-9777-86839ca484e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:11.781778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1265/2512] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:11 2022] DELETE /identity/v3/projects/f52d0d7793854837a134c7e9c54b690e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:11.786911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.787389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.787503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.808013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:11.809215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:11.871814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1248/2513] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:11.880123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.880403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.880668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.902767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:11.903963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:11.913377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1266/2514] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:11.918028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:11.928693 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe38240-f1c5-48a1-b304-1ed2b91aeaad None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:11.934535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:11.937049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.937353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.937669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.938338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:13:11.938645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:11.940259 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:11.952313 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1249/2515] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:13:11.956666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:11.966212 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5250ee-bc0d-49ac-91c1-0b7f7dc6299f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:11.972141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:11.972735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:11.973004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:11.973264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:11.973919 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:11.975851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.006941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1267/2516] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:11 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:12.011531 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.021894 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403424c1-553e-49dc-a520-9b262df27ca7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.047429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:12.081337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.081993 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.082269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.082535 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.083152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.085061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.108725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1250/2517] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:12.113051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.123837 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5781-9e2d-42ac-9939-13a7a33f7913 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.129409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.130041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.130316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.130573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.131256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.133070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.143689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1268/2518] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:12.147901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.158059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5708-d22e-4b5f-8596-401ae3241c9e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.163563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.164158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.164435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.164702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.165583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.167277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b681-0b7e-48cd-b9a6-77c12187757e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.179510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1251/2519] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:12.183817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.195277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d2ad-91ba-457f-ad28-17295231a401 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.200448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.201088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.201360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.201692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] PATH_INFO: `/v3/projects/87406ea1089646c5ac14e8c036600166/users/2ba8e80cdbdf4a83b93e5440366c8d56/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.216396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87406ea1089646c5ac14e8c036600166, user_id=2ba8e80cdbdf4a83b93e5440366c8d56, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.219097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.233636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1269/2520] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:12 2022] PUT /identity/v3/projects/87406ea1089646c5ac14e8c036600166/users/2ba8e80cdbdf4a83b93e5440366c8d56/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:12.244741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.245026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.245299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.272603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] MFA Rules not processed for user `2ba8e80cdbdf4a83b93e5440366c8d56`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:12.273764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:12.339398 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1252/2521] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:12.343868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.418860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66372f-a33a-45db-916f-d1b89341963d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.423514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.424147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.424422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.424707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.425674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.427473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.460055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1270/2522] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:12.461416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.473057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8ee92-3979-4145-8b47-6c6de3a17e01 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.539938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.540482 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.540765 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.541037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.541697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.543649 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.572776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1253/2523] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:12.574113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.585251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3db5ab-b8c6-47f8-8ced-2fde162fde62 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.590378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.590982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.591257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.591530 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.592198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.593848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.604894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1271/2524] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:12.609194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.620486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced90d34-3605-4ad8-b42a-7dee47066440 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.626163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.626775 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.627042 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.627306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.628000 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.629680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123908d9-1ac6-4434-90a0-ecf3f6649b9e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.640462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1254/2525] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:12.645305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.655533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ada1d2-d271-4f3e-a58e-4cf5956bb648 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.660504 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.661153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.661427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.661726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] PATH_INFO: `/v3/projects/f02dffda608547df96814bee6d797887/users/846b0a067cf94e37826216fe8ade15c4/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.678050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02dffda608547df96814bee6d797887, user_id=846b0a067cf94e37826216fe8ade15c4, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.681101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.692027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:12.696924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1272/2526] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:12 2022] PUT /identity/v3/projects/f02dffda608547df96814bee6d797887/users/846b0a067cf94e37826216fe8ade15c4/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:12.708317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.709132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.709132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.740569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] MFA Rules not processed for user `846b0a067cf94e37826216fe8ade15c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:12.741848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:12.745975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:12.811674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1255/2527] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:12.816521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.882236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbffd04-da78-4e80-b2a5-15726ee9ed35 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:12.887408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:12.888016 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:12.888291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:12.888558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:12.889175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:12.891123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:12.923024 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1273/2528] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:13:12.927428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:12.939013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a255289-c023-41e4-8fc9-79c486036863 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.003800 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.004447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.004723 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.004983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.005684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.007608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.036720 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1256/2529] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:12 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:13.042354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.052848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8647-3c83-4467-8033-c63234930f7c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.058403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.059001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.059270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.059535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.060177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.061804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.073861 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1274/2530] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:13.078262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.089505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e676d-1859-43b7-8115-aedb1c55719b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.094748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.095402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.095679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.095953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] PATH_INFO: `/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250/users/0031b042b1174ba48b303444a88faa34/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.113211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed31e21a98c4eb9b0f5fc77177d7250, user_id=0031b042b1174ba48b303444a88faa34, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.115900 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.133008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1257/2531] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:13 2022] PUT /identity/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250/users/0031b042b1174ba48b303444a88faa34/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:13.137007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.149689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85bfb6-ca87-4054-b143-92f098114df5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.215163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.215765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.216039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.216302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.216941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.218610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fbb61-c1af-4919-b6b2-853b85617550 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.229122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1275/2532] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:13.234969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.245807 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019ecc4-468d-491b-810f-af36f3a98779 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.250897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.251551 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.251826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.252091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] PATH_INFO: `/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250/users/0031b042b1174ba48b303444a88faa34/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.254190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed31e21a98c4eb9b0f5fc77177d7250, user_id=0031b042b1174ba48b303444a88faa34, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.256811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.268549 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1258/2533] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:13 2022] PUT /identity/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250/users/0031b042b1174ba48b303444a88faa34/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:13.279785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.280141 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.280518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.316727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] MFA Rules not processed for user `0031b042b1174ba48b303444a88faa34`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:13.317935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:13.383100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:13.394007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1276/2534] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:13.408623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.410730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:13.469372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478d1da-b9d0-4ad6-b26c-4a7a433b5d6f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.473379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.474007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.474280 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.474543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.484009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.485420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.490370 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:13.495246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1259/2535] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:13.542786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:13.544236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:13.558362 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affb6f0-0f95-417a-a305-b6db7ca67833 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:13.562257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:13.562913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:13.563158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:13.563433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:13.621686 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:13.623178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6251249-c984-47a7-b644-f7ffaa5adc46 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:13.632582 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1277/2536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:16.563874 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:16.630947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8671d-ff9f-4ae5-b97b-a14ce7d56835 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:16.637015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:16.637655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:16.637966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:16.638236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] PATH_INFO: `/v3/users/2ba8e80cdbdf4a83b93e5440366c8d56` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:16.639345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba8e80cdbdf4a83b93e5440366c8d56)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:16.641274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:16.691784 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba8e80cdbdf4a83b93e5440366c8d56'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:16.692445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba8e80cdbdf4a83b93e5440366c8d56'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:16.693101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba8e80cdbdf4a83b93e5440366c8d56'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:16.705494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba8e80cdbdf4a83b93e5440366c8d56'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:16.712533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba8e80cdbdf4a83b93e5440366c8d56'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:16.729464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1260/2537] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:16 2022] DELETE /identity/v3/users/2ba8e80cdbdf4a83b93e5440366c8d56 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:16.734303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:16.734793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:16.734910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:16.753810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:16.755013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:16.814874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1278/2538] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:13:16.827311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:16.828749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:16.892943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f5b9d-1881-445f-aba3-6aa1b35da632 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:16.897882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:16.898506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:16.898782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:16.899034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:16.909275 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:16.910747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:16.920856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1261/2539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:17.267399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:17.269478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:17.310773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554425c4-5be4-472a-ad71-9180e7c57f19 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:17.316824 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:17.318007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:17.318370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:17.318619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] PATH_INFO: `/v3/projects/87406ea1089646c5ac14e8c036600166` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:17.319798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87406ea1089646c5ac14e8c036600166)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:17.321758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:17.374711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87406ea1089646c5ac14e8c036600166', 'request_id': 'req-09411f13-8c66-4560-bdf5-22fd59572610'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.380423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87406ea1089646c5ac14e8c036600166', 'request_id': 'req-09411f13-8c66-4560-bdf5-22fd59572610'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.386763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87406ea1089646c5ac14e8c036600166', 'request_id': 'req-09411f13-8c66-4560-bdf5-22fd59572610'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.400579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] The token cache is being invalidate because project 87406ea1089646c5ac14e8c036600166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:17.400871 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.401513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.403046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1279/2540] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:17 2022] DELETE /identity/v3/projects/87406ea1089646c5ac14e8c036600166 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:17.407097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:17.416956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f9719-29ff-4aba-9ebb-e71b6ca2957c service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:17.435553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:17.477244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:17.477924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:17.478207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:17.478479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] PATH_INFO: `/v3/users/846b0a067cf94e37826216fe8ade15c4` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:17.479756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=846b0a067cf94e37826216fe8ade15c4)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:17.481713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:17.529699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '846b0a067cf94e37826216fe8ade15c4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.530455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '846b0a067cf94e37826216fe8ade15c4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.530974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '846b0a067cf94e37826216fe8ade15c4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.544582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '846b0a067cf94e37826216fe8ade15c4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.551839 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '846b0a067cf94e37826216fe8ade15c4'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.568552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1262/2541] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:17 2022] DELETE /identity/v3/users/846b0a067cf94e37826216fe8ade15c4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:13:17.857246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:17.859311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:17.873338 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09411f13-8c66-4560-bdf5-22fd59572610 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:17.886035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:17.934516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:17.935161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:17.935432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:17.935697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] PATH_INFO: `/v3/projects/f02dffda608547df96814bee6d797887` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:17.936819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02dffda608547df96814bee6d797887)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:17.938793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:17.988627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02dffda608547df96814bee6d797887', 'request_id': 'req-93a2ee69-c458-46be-9f5d-223d30422c57'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:17.996083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02dffda608547df96814bee6d797887', 'request_id': 'req-93a2ee69-c458-46be-9f5d-223d30422c57'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.003085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02dffda608547df96814bee6d797887', 'request_id': 'req-93a2ee69-c458-46be-9f5d-223d30422c57'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.018929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] The token cache is being invalidate because project f02dffda608547df96814bee6d797887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:18.020032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.020663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.022347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1280/2542] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:17 2022] DELETE /identity/v3/projects/f02dffda608547df96814bee6d797887 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:18.027558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:18.029132 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:18.042321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7b091-7b99-4dcf-8ad7-2e9468995d7e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:18.109975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:18.110622 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.110907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.111182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] PATH_INFO: `/v3/users/0031b042b1174ba48b303444a88faa34` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.112389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0031b042b1174ba48b303444a88faa34)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:18.114416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:18.167033 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:18.169848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0031b042b1174ba48b303444a88faa34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.170579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0031b042b1174ba48b303444a88faa34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.171446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0031b042b1174ba48b303444a88faa34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.184724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0031b042b1174ba48b303444a88faa34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.191432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-2cdd3684-9b16-465d-8442-ab6af99d1351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0031b042b1174ba48b303444a88faa34'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.206648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1263/2543] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:18 2022] DELETE /identity/v3/users/0031b042b1174ba48b303444a88faa34 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:18.568951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:18.570498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:18.584399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2ee69-c458-46be-9f5d-223d30422c57 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:18.649613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:18.650225 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.650499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.650768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] PATH_INFO: `/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.652025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed31e21a98c4eb9b0f5fc77177d7250)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:18.654091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:18.704471 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed31e21a98c4eb9b0f5fc77177d7250', 'request_id': 'req-84335740-6448-42b2-a960-a8752219db59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.705661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:18.711042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed31e21a98c4eb9b0f5fc77177d7250', 'request_id': 'req-84335740-6448-42b2-a960-a8752219db59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.718745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed31e21a98c4eb9b0f5fc77177d7250', 'request_id': 'req-84335740-6448-42b2-a960-a8752219db59'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.734377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] The token cache is being invalidate because project bed31e21a98c4eb9b0f5fc77177d7250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:18.734665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.735308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-84335740-6448-42b2-a960-a8752219db59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:18.736900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1281/2544] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:18 2022] DELETE /identity/v3/projects/bed31e21a98c4eb9b0f5fc77177d7250 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:18.744136 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.744619 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.744715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.765228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:18.766474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:18.828655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1264/2545] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:13:18.837499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.837882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.838200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.860004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:18.861226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:18.874593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1282/2546] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:18.875138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:18.885106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5c4ff-3f20-4115-8920-bb8568fd0663 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:18.890196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:18.890804 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.891059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.891323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.891992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:13:18.892285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:18.893906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:18.905954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1265/2547] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:13:18.910437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:18.919882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82399b-d003-4a38-9d28-ffffe07eef77 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:18.924649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:18.925247 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:18.925519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:18.925820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:18.926454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:18.928794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:18.960340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1283/2548] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:18 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:18.965323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:18.976191 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61c3f-65c8-4cd0-9da7-a727b8255478 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:19.036827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:19.037456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.037791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.038068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.038707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:19.040692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:19.063973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1266/2549] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:18 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:13:19.068421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:19.078888 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a401b6-ae11-4a60-9386-329be2f70988 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:19.084565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:19.085219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.085495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.085763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.086409 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:19.088030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:19.098715 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1284/2550] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:19.103225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:19.118963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52b320-e7b2-4c02-bc37-911eb57c498d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:19.125170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:19.125976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.126258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.126524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.127160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:19.128758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c7ff3-7893-43ca-b5e7-426f4bcdaba9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:19.140234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1267/2551] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:19.144580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:19.155226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72cf9b1-cb94-48ab-80a4-447938f58bc3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:19.160143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:19.160875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.161135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.161404 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] PATH_INFO: `/v3/projects/df373a19891946e3a9f9a0a0ec7e8ad1/users/5b6f6e3e6a7744a39c06fd0b901b63f2/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.176677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df373a19891946e3a9f9a0a0ec7e8ad1, user_id=5b6f6e3e6a7744a39c06fd0b901b63f2, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:19.179760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:19.195293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1285/2552] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:19 2022] PUT /identity/v3/projects/df373a19891946e3a9f9a0a0ec7e8ad1/users/5b6f6e3e6a7744a39c06fd0b901b63f2/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:19.206563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1892-0900-488e-bcc7-88dcd53f754b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.206847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1892-0900-488e-bcc7-88dcd53f754b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.207119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b1892-0900-488e-bcc7-88dcd53f754b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.237168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-332b1892-0900-488e-bcc7-88dcd53f754b None None] MFA Rules not processed for user `5b6f6e3e6a7744a39c06fd0b901b63f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:19.238563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-332b1892-0900-488e-bcc7-88dcd53f754b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:19.253194 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:19.318196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1268/2553] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:19.326726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:19.329070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:19.394490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ddb54-337f-4cff-90d0-816cc3f4c1b1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:19.398519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:19.399147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:19.399425 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:19.399692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:19.408149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:19.409651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:19.418599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1286/2554] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:22.005610 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:22.006210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:22.006210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:22.069385 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:22.070655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:22.158567 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1269/2555] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:13:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:13:22.173284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:22.174796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:22.262845 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:22.273744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42c3b4-5f79-4f25-b3df-792d4b5abe75 service glance] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:22.279333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:22.279965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:22.280237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:22.280500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:22.290783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:22.292381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:22.301109 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1287/2556] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:13:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:13:27.917109 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:27.918674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:27.978638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f086bbc-16f7-4141-a184-bebd7856f7f1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:27.982566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:27.983176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:27.983448 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:27.983716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:27.985614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:28.029294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:28.052208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:28.054058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:28.066682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1270/2557] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:31.236322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:31.237895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:31.259807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be323fdd-9383-4d0d-954e-5d68e4a02a29 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:31.264554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:31.265198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.265476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.265781 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] PATH_INFO: `/v3/users/454c2bf24ed94f82b453df36e6e8eac1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.280414 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] RBAC: Authorizing `identity:delete_user(user_id=454c2bf24ed94f82b453df36e6e8eac1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:31.282448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:31.331325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454c2bf24ed94f82b453df36e6e8eac1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.338432 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454c2bf24ed94f82b453df36e6e8eac1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.350786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454c2bf24ed94f82b453df36e6e8eac1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.359662 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454c2bf24ed94f82b453df36e6e8eac1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.360335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2ef0d061-3457-4de5-a51f-77e25dc9113e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454c2bf24ed94f82b453df36e6e8eac1'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.366394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1288/2558] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:31 2022] DELETE /identity/v3/users/454c2bf24ed94f82b453df36e6e8eac1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:31.546827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:31.558464 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc1ecd-7f55-4242-9a55-c2ee559e1880 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:31.618297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:31.618931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.619207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.619478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] PATH_INFO: `/v3/projects/56d236fd14cd49168a3842369badc524` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.629556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56d236fd14cd49168a3842369badc524)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:31.631499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:31.680142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56d236fd14cd49168a3842369badc524', 'request_id': 'req-989c4003-b27a-4241-b260-78fdae4f3ac8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.685668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56d236fd14cd49168a3842369badc524', 'request_id': 'req-989c4003-b27a-4241-b260-78fdae4f3ac8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.694786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56d236fd14cd49168a3842369badc524', 'request_id': 'req-989c4003-b27a-4241-b260-78fdae4f3ac8'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.706709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] The token cache is being invalidate because project 56d236fd14cd49168a3842369badc524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:31.707003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.707675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-989c4003-b27a-4241-b260-78fdae4f3ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:31.709202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1271/2559] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:31 2022] DELETE /identity/v3/projects/56d236fd14cd49168a3842369badc524 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:31.714253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.714539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.714906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.734672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:31.735821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:31.797401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1289/2560] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:31.806069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.806359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.806630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.827366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:31.828632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:31.837650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1272/2561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:31.842099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:31.852188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018c5da-1205-4873-b4a4-519b86d4b510 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:31.857569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:31.858153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.858447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.858717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.859366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:13:31.859665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:31.861281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:31.873282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1290/2562] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:13:31.877802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:31.888243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a0a9a-5511-4249-9a4e-67077beceb4e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:31.893510 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:31.894146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:31.894420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:31.894686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:31.895336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:31.897226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:31.927224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1273/2563] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:31 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:31.931874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:31.942130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8335c71-a8a4-4c83-8a6d-bc1326d25a88 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:31.975988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:32.003542 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.004104 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.004378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.004668 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.005282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.007307 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.036243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1291/2564] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:31 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:32.037651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.048799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a280ac-6297-40a9-9c1e-1d4a15a58deb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.054063 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.054663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.054934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.055199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.055835 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.057444 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.067767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1274/2565] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:32.071803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.082900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462560d5-0e02-4dd8-a560-77324a73be88 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.088013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.088609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.088887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.089158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.089835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.091436 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014a003-96df-4f5b-9f49-e3ec1a78d21f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.103291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1292/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:32.107737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.118374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7646d7-316a-4d15-b296-8bc5514d4db2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.124488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.125144 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.125424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.125733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] PATH_INFO: `/v3/projects/c6dc754766694a5d8c7a43db3f8d6e66/users/6a4753dd912b4bc68ed3a45282effd9e/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.141437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6dc754766694a5d8c7a43db3f8d6e66, user_id=6a4753dd912b4bc68ed3a45282effd9e, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.144119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.147242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:32.162350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1275/2567] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:32 2022] PUT /identity/v3/projects/c6dc754766694a5d8c7a43db3f8d6e66/users/6a4753dd912b4bc68ed3a45282effd9e/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:32.172802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.173085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.173354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.204180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] MFA Rules not processed for user `6a4753dd912b4bc68ed3a45282effd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:32.205396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:32.272274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1293/2568] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:13:32.276869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.339315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0706a-386f-4487-b7fc-e0d5f32b9930 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.344604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.345207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.345483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.345787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.346422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.348319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.376675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1276/2569] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:32 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:13:32.381496 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.397798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512aece-2e2e-491e-b8ea-47be8a4b0c15 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.458158 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.458742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.459099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.459370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.459996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.461990 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.485162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1294/2570] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:32 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:32.489657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.500554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28d5d4-a617-403a-81cd-cbb529d3a961 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.506475 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.507080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.507311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.507607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.508246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.509893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.521089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1277/2571] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:32.525457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.536075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e81-d645-40c9-b036-27e3f468b2d2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.541132 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.541979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.542325 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.542669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] PATH_INFO: `/v3/projects/0beed4f453494662ba2d2aa010665da0/users/a93e6724d18b4020914c514b389f30e7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.560580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0beed4f453494662ba2d2aa010665da0, user_id=a93e6724d18b4020914c514b389f30e7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.563281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.582941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1295/2572] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:32 2022] PUT /identity/v3/projects/0beed4f453494662ba2d2aa010665da0/users/a93e6724d18b4020914c514b389f30e7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:32.587142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.597308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0e793-3365-49ba-a474-6f0c8398a61c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.664802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.665413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.665738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.666020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.666659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.668269 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2522-171f-4f07-98f3-028d8aa3aeb7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.678044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1278/2573] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:32.682890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.694140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb2e5-4710-484b-9c39-7c6190ee3061 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.698849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.699490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.699756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.700020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] PATH_INFO: `/v3/projects/0beed4f453494662ba2d2aa010665da0/users/a93e6724d18b4020914c514b389f30e7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.702078 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0beed4f453494662ba2d2aa010665da0, user_id=a93e6724d18b4020914c514b389f30e7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.704706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.708220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:32.718146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1296/2574] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:32 2022] PUT /identity/v3/projects/0beed4f453494662ba2d2aa010665da0/users/a93e6724d18b4020914c514b389f30e7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:32.730812 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.731211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.731578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.761130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] MFA Rules not processed for user `a93e6724d18b4020914c514b389f30e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:32.762575 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:32.814487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:32.835070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1279/2575] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:32.846483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:32.848392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:32.906978 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4c79f-b7d3-4f02-aa96-5267c942f1f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:32.910924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:32.911525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:32.911795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:32.912040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:32.922673 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:32.924179 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:32.934682 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1297/2576] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:33.012310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:33.022968 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685afe-04df-4dac-8ff3-175daf5be924 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:33.027243 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:33.027848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:33.028127 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:33.028403 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:33.101297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:33.102761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcf2c8-d2c8-4e53-9d53-b8a6ce6e14be service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:33.112513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1280/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:42.930464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=28 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:43.107877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=32 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:46.725365 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:46.791941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f922b-1562-40fd-bf6c-97876ead3a1b service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:46.803545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:46.804270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:46.805220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:46.805744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] PATH_INFO: `/v3/users/6a4753dd912b4bc68ed3a45282effd9e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:46.808149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4753dd912b4bc68ed3a45282effd9e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:46.811138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:46.878198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4753dd912b4bc68ed3a45282effd9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:46.885265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4753dd912b4bc68ed3a45282effd9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:46.897186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4753dd912b4bc68ed3a45282effd9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:46.907093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4753dd912b4bc68ed3a45282effd9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:46.907701 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4753dd912b4bc68ed3a45282effd9e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:46.913666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1298/2578] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:46 2022] DELETE /identity/v3/users/6a4753dd912b4bc68ed3a45282effd9e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:46.918238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:46.918771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:46.918870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:46.939390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:46.940578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:47.000119 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1281/2579] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:47.010074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:47.011600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:47.080046 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56809ee0-4ac6-4492-b929-560895510451 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:47.084203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:47.084826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:47.085202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:47.085469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:47.094942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:47.096380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:47.106566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1299/2580] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:47.400049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:47.401560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:47.415329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9d0e1-e0d2-418c-91ed-cb4cc33e6080 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:47.420407 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:47.421048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:47.421327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:47.421616 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] PATH_INFO: `/v3/projects/c6dc754766694a5d8c7a43db3f8d6e66` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:47.429671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6dc754766694a5d8c7a43db3f8d6e66)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:47.431660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:47.479250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6dc754766694a5d8c7a43db3f8d6e66', 'request_id': 'req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.484707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6dc754766694a5d8c7a43db3f8d6e66', 'request_id': 'req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.494090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6dc754766694a5d8c7a43db3f8d6e66', 'request_id': 'req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.505147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] The token cache is being invalidate because project c6dc754766694a5d8c7a43db3f8d6e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:47.505430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.506106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.507941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1282/2581] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:47 2022] DELETE /identity/v3/projects/c6dc754766694a5d8c7a43db3f8d6e66 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:47.512040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:47.521841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb317-bf67-4b42-b622-5491ffd98734 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:47.541394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:47.584353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:47.584988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:47.585257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:47.585525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] PATH_INFO: `/v3/users/a93e6724d18b4020914c514b389f30e7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:47.586819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a93e6724d18b4020914c514b389f30e7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:47.588784 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:47.647017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a93e6724d18b4020914c514b389f30e7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.654963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a93e6724d18b4020914c514b389f30e7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.667345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a93e6724d18b4020914c514b389f30e7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.676485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a93e6724d18b4020914c514b389f30e7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.677092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7d015cc6-b76d-4f2a-8d51-8f484c3c4613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a93e6724d18b4020914c514b389f30e7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:47.683628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1300/2582] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:13:47 2022] DELETE /identity/v3/users/a93e6724d18b4020914c514b389f30e7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:47.966838 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:47.968440 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:47.985478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95251f-4bbc-44ce-a4e3-cf9dcf6958be admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.052947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.053881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.054159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.054432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] PATH_INFO: `/v3/projects/0beed4f453494662ba2d2aa010665da0` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.055586 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0beed4f453494662ba2d2aa010665da0)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.057491 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.090419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:48.114877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0beed4f453494662ba2d2aa010665da0', 'request_id': 'req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:48.120472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0beed4f453494662ba2d2aa010665da0', 'request_id': 'req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:48.129279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0beed4f453494662ba2d2aa010665da0', 'request_id': 'req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:48.140866 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] The token cache is being invalidate because project 0beed4f453494662ba2d2aa010665da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:13:48.141200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:48.141898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-e0182f00-8ef8-43a3-83d1-99a1be1be5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:13:48.143421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1283/2583] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:13:47 2022] DELETE /identity/v3/projects/0beed4f453494662ba2d2aa010665da0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:48.148692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.149166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.149288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.168894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:48.170034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:48.238000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1301/2584] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:48.246653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.246934 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.247207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.267218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:48.268413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:48.279384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1284/2585] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:48.283800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.293889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd594c08-89b8-4025-bc12-b357291e5b64 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.300215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.300811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.301076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.301337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.302007 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:13:48.302360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.304370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.316095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1302/2586] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 17 20:13:48.320373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.329983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede7678-8af1-4ac1-a165-92fb39d0e95c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.335965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.336559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.336834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.337091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.337749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.339701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.370698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1285/2587] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:48.374980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.385420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fec28-2e26-4bb7-bc41-224be0def077 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.413807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:48.449760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.450329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.450624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.450862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.451513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.453431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.475421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1303/2588] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:13:48.479670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.490185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc2583-5e03-4284-a1c7-a2d1cbeb8374 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.495695 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.496301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.496577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.496847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.497487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.499110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.509802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1286/2589] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:13:48.514077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.524372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5027c-c975-4caa-8c98-a0013c40d5ac admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.529866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.530442 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.530711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.530973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.531614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.533192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2f35f-7be1-41a7-b30a-c548237aba91 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.544219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1304/2590] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:48.549207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.565522 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641deeba-d2a9-41fb-8c32-e8c4be190118 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.571210 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.571880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.572163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.572880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] PATH_INFO: `/v3/projects/0fe61e601ccc4ee39ac92b8f20e8a134/users/3565af44055942abb5c1a59f0f834dcf/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.588057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe61e601ccc4ee39ac92b8f20e8a134, user_id=3565af44055942abb5c1a59f0f834dcf, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.590767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.608062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1287/2591] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:48 2022] PUT /identity/v3/projects/0fe61e601ccc4ee39ac92b8f20e8a134/users/3565af44055942abb5c1a59f0f834dcf/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:48.618633 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.618941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.619211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.647962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] MFA Rules not processed for user `3565af44055942abb5c1a59f0f834dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:48.649199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:48.716060 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1305/2592] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:13:48.720724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.799948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407edbc8-2e6b-4ad0-bb25-3361f5625177 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.806856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.807465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.807739 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.808026 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.808648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.810600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.847465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1288/2593] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:48.851809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.866548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af0fb0-b050-46a6-8ac6-a8c5156ce0b5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.935899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.936501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.936773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.937035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.937693 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.939706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.963027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1306/2594] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:13:48 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:13:48.967623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:48.978689 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aa0e4-d117-4ef7-895c-02b31231c605 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:48.983722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:48.984321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:48.984603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:48.984879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:48.985529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:48.987171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:48.998435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1289/2595] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:49.004868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:49.021516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086a6c1-3e86-40c4-9aae-2aac115ee643 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:49.027378 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:49.028031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:49.028303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:49.028567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] PATH_INFO: `/v3/projects/a5825bca11694c9699c39b3b47aed003/users/5ff3a4e0917c403abab312b619804a4f/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:49.045468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5825bca11694c9699c39b3b47aed003, user_id=5ff3a4e0917c403abab312b619804a4f, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:49.048165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:49.063626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1307/2596] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:49 2022] PUT /identity/v3/projects/a5825bca11694c9699c39b3b47aed003/users/5ff3a4e0917c403abab312b619804a4f/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:49.068059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:49.078203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec36fcc-ead9-44c3-9572-d292a3724faf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:49.099130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:49.141711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:49.142601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:49.143009 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:49.143425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:49.144455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:49.147104 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419b037-1a17-41cf-8f5f-b9795942ab59 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:49.162406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1290/2597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:13:49.166762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:49.181437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:13:49.191286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767c57-19d8-4bc2-824e-adbf77e0f833 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:49.197303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:49.198001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:49.198304 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:49.198570 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] PATH_INFO: `/v3/projects/a5825bca11694c9699c39b3b47aed003/users/5ff3a4e0917c403abab312b619804a4f/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:49.200724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5825bca11694c9699c39b3b47aed003, user_id=5ff3a4e0917c403abab312b619804a4f, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:49.203419 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:49.213988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1308/2598] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:13:49 2022] PUT /identity/v3/projects/a5825bca11694c9699c39b3b47aed003/users/5ff3a4e0917c403abab312b619804a4f/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:13:49.225774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:49.226062 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:49.226340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:49.258138 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] MFA Rules not processed for user `5ff3a4e0917c403abab312b619804a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:13:49.259362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:49.338240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1291/2599] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:13:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:13:49.349792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:49.351611 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:49.410495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae8465-ebd2-48aa-b4f6-5494c220e39b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:49.414700 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:49.415302 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:49.415580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:49.415846 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:49.479671 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:49.481111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:49.490207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1309/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:58.363941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:13:58.365664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:13:58.378706 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c12202-b216-4e49-bcc0-110cc3076f3d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:13:58.383235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:13:58.383841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:13:58.384116 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:13:58.384389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:13:58.394459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:13:58.395872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:13:58.411666 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1292/2601] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:13:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:13:59.486435 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:03.638476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:03.640097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:03.701789 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8a75-ea2f-441b-8722-c3718f3ed433 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:03.705965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:03.706565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:03.706838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:03.707103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:03.709107 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:03.719229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:03.720664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1134c3c-94c3-40be-bbf0-332cdaae81c4 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:03.728735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1310/2602] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:05.293722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:05.295183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:05.369396 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fef2f0-2adf-4b14-a0b5-91ec81b5f164 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:05.375187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:05.375908 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:05.376302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:05.376640 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] PATH_INFO: `/v3/users/5b6f6e3e6a7744a39c06fd0b901b63f2` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:05.382035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6f6e3e6a7744a39c06fd0b901b63f2)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:05.382035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:05.421979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:05.433453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6f6e3e6a7744a39c06fd0b901b63f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:05.434256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6f6e3e6a7744a39c06fd0b901b63f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:05.434936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6f6e3e6a7744a39c06fd0b901b63f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:05.450463 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6f6e3e6a7744a39c06fd0b901b63f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:05.457761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6f6e3e6a7744a39c06fd0b901b63f2'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:05.476527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1293/2603] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:05 2022] DELETE /identity/v3/users/5b6f6e3e6a7744a39c06fd0b901b63f2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:05.482608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:05.483120 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:05.483218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:05.502661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:05.503907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:05.564660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1311/2604] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:05.574724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:05.576327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:05.634597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565e9c5-6546-4d8f-9f00-a6f3abddce14 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:05.638325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:05.638990 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:05.639240 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:05.639533 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:05.649389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:05.651071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166bbe9-6840-422a-aefb-91da471862a2 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:05.661238 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1294/2605] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:05.948965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:05.950626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:05.963208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361236-b962-40c3-9806-dad3b47b960d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:05.968743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:05.969411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:05.969718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:05.969981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] PATH_INFO: `/v3/projects/df373a19891946e3a9f9a0a0ec7e8ad1` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:05.971222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df373a19891946e3a9f9a0a0ec7e8ad1)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:05.973162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.021771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df373a19891946e3a9f9a0a0ec7e8ad1', 'request_id': 'req-86c39622-d9cc-4cc0-9d4f-2398eef905bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:06.027136 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df373a19891946e3a9f9a0a0ec7e8ad1', 'request_id': 'req-86c39622-d9cc-4cc0-9d4f-2398eef905bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:06.033180 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df373a19891946e3a9f9a0a0ec7e8ad1', 'request_id': 'req-86c39622-d9cc-4cc0-9d4f-2398eef905bb'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:06.048635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] The token cache is being invalidate because project df373a19891946e3a9f9a0a0ec7e8ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:06.048921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:06.049563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-86c39622-d9cc-4cc0-9d4f-2398eef905bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:06.051115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1312/2606] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:05 2022] DELETE /identity/v3/projects/df373a19891946e3a9f9a0a0ec7e8ad1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:06.056700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.057012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.057283 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.076903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:06.078137 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:06.140568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1295/2607] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:06.149357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.149709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.150062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.176967 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:06.178744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:06.188153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1313/2608] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:06.192563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.202917 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8b659-5abc-4196-a208-85d7fff4cf51 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.209278 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.209930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.210205 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.210474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.211120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:06.211417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.212999 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.225525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1296/2609] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:06.230099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.239619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284ea2-063c-4a63-aa13-01872f8a5da3 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.245940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.246627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.246890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.247152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.247780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.249746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.276370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:06.287531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1314/2610] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:06 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:06.291161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.301569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bff19e-a2cc-479e-afa7-33b509684ac6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.343274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:06.362239 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.362846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.363120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.363389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.364007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.367953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.392085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1297/2611] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:06 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:06.396571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.407550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49e25b-9ae7-4f57-b068-3d125e601881 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.412513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.413303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.413571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.413860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.414502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.416146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.426324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1315/2612] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:06.430865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.441747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce18-0495-419c-ac39-c8921ae289bd admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.447670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.448299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.448578 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.448840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.449518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.451152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3131821-b19d-4e42-9c0f-c15348ab9840 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.462935 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1298/2613] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:06.468168 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.479346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d21519-600c-46cf-aa76-b8d29c2b707e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.485875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.486484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.486780 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.487041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] PATH_INFO: `/v3/projects/b4c690be95794f15913662bf3c709407/users/434b712c217a47b79ecd966efbd8428b/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.502985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c690be95794f15913662bf3c709407, user_id=434b712c217a47b79ecd966efbd8428b, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.505636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.527610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1316/2614] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:06 2022] PUT /identity/v3/projects/b4c690be95794f15913662bf3c709407/users/434b712c217a47b79ecd966efbd8428b/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:06.538288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.538572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.538844 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.571129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] MFA Rules not processed for user `434b712c217a47b79ecd966efbd8428b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:06.572360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:06.646233 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1299/2615] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:06.657222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:06.659083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:06.717908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda95636-1829-4b9d-bd38-f69973b31602 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:06.722218 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:06.722829 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:06.723115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:06.723380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:06.733005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:06.734525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6192c-f06e-42ab-bcf1-872f2e7a3c58 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:06.743428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1317/2616] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:16.643208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=47,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:16.739305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:39.052302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:39.054413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:39.122402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b75fe-ecbd-4a13-a91d-5830287dde4e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:39.128175 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:39.128847 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:39.129162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:39.129408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] PATH_INFO: `/v3/users/3565af44055942abb5c1a59f0f834dcf` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:39.130673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3565af44055942abb5c1a59f0f834dcf)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:39.132781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:39.182178 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3565af44055942abb5c1a59f0f834dcf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.182828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3565af44055942abb5c1a59f0f834dcf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.183439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3565af44055942abb5c1a59f0f834dcf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.195556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3565af44055942abb5c1a59f0f834dcf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.202207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3565af44055942abb5c1a59f0f834dcf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.216417 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1300/2617] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:39 2022] DELETE /identity/v3/users/3565af44055942abb5c1a59f0f834dcf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:39.221339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:39.221901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:39.221901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:39.240623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:39.241847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:39.304669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1318/2618] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:39.315255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:39.316971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:39.378912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99103dea-3203-49ef-a0ec-fdd27ed6830c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:39.383697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:39.384314 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:39.384585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:39.384849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:39.396222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:39.398125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:39.407984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1301/2619] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:39.687294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:39.688807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:39.703847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9543f-b5cf-4595-af17-42439df8eeff None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:39.710505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:39.711186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:39.711397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:39.711667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] PATH_INFO: `/v3/projects/0fe61e601ccc4ee39ac92b8f20e8a134` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:39.713067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe61e601ccc4ee39ac92b8f20e8a134)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:39.715112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:39.767134 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe61e601ccc4ee39ac92b8f20e8a134', 'request_id': 'req-7fa2d558-e911-44cd-85df-eba592a03da0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.772696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe61e601ccc4ee39ac92b8f20e8a134', 'request_id': 'req-7fa2d558-e911-44cd-85df-eba592a03da0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.778925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe61e601ccc4ee39ac92b8f20e8a134', 'request_id': 'req-7fa2d558-e911-44cd-85df-eba592a03da0'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.792270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] The token cache is being invalidate because project 0fe61e601ccc4ee39ac92b8f20e8a134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:39.792559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.793186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:39.794706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1319/2620] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:39 2022] DELETE /identity/v3/projects/0fe61e601ccc4ee39ac92b8f20e8a134 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:39.798845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:39.808834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33adc-2dd9-43e0-9f58-c445f62976c5 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:39.827018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:39.995768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:39.996358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:39.996642 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:39.996916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] PATH_INFO: `/v3/users/5ff3a4e0917c403abab312b619804a4f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:39.998211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff3a4e0917c403abab312b619804a4f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.000182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.050595 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff3a4e0917c403abab312b619804a4f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.051499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff3a4e0917c403abab312b619804a4f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.051861 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff3a4e0917c403abab312b619804a4f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.064727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff3a4e0917c403abab312b619804a4f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.071880 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-6870aabe-1c5a-426c-8559-dce6f468aa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff3a4e0917c403abab312b619804a4f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.085922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1302/2621] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:39 2022] DELETE /identity/v3/users/5ff3a4e0917c403abab312b619804a4f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:40.366607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.368182 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:40.381820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2d558-e911-44cd-85df-eba592a03da0 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.440979 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.441636 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.441910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.442176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] PATH_INFO: `/v3/projects/a5825bca11694c9699c39b3b47aed003` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.443310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5825bca11694c9699c39b3b47aed003)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.445208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.480090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:40.499105 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5825bca11694c9699c39b3b47aed003', 'request_id': 'req-58dfe01b-e86e-4ae6-b916-8e15a781777f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.505020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5825bca11694c9699c39b3b47aed003', 'request_id': 'req-58dfe01b-e86e-4ae6-b916-8e15a781777f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.511519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5825bca11694c9699c39b3b47aed003', 'request_id': 'req-58dfe01b-e86e-4ae6-b916-8e15a781777f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.526357 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] The token cache is being invalidate because project a5825bca11694c9699c39b3b47aed003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:40.526648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.527330 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-58dfe01b-e86e-4ae6-b916-8e15a781777f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.528943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1320/2622] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:40 2022] DELETE /identity/v3/projects/a5825bca11694c9699c39b3b47aed003 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:40.534231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.534521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.534790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.555261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:40.556477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:40.616963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1303/2623] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:40.625128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.625410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.625718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.648097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:40.649340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:40.659053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1321/2624] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:40.663766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.674488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6ee9b-fe32-44d7-933e-b2409e83ed5c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.680713 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.681329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.681638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.681906 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.682550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:40.682848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.684422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.696395 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1304/2625] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:40.702424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.711801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41416f-aff2-44dc-a611-619c0ff6339a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.717568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.718201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.718579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.718808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.719468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.721397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.753246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1322/2626] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:40.757686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.768044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70560163-e291-49d6-856b-5665957a20cb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.793753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.795446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:40.795925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:40.832895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.833627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.833910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.834184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.834828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.836729 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.860018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1305/2627] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:40.861977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2cdb0-d4d0-4066-8c64-bf67ff69e4d2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.864208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.866651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.867346 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.867562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.867831 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] PATH_INFO: `/v3/users/434b712c217a47b79ecd966efbd8428b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.868996 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434b712c217a47b79ecd966efbd8428b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.870962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.873613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84af21f-ef94-4adb-be81-6ff443c7af46 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.878174 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.878754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.879030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.879299 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.879947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.881495 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.891769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1306/2628] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:40.895803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.905870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0107ddd-d6cf-4b82-9fc6-7535f9251f22 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.911445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.912036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.912303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.912572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.913204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.914816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.921317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434b712c217a47b79ecd966efbd8428b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.925228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1307/2629] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:40.929789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:40.931849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434b712c217a47b79ecd966efbd8428b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.940048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586c4f9-8671-4300-a280-e0cb9875b45a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:40.944430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434b712c217a47b79ecd966efbd8428b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.945235 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:40.945929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.946179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.946606 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] PATH_INFO: `/v3/projects/3875bd47e8f748b68e985bb095c08339/users/96b0028d546b441495e246bc434a4c7e/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.953720 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434b712c217a47b79ecd966efbd8428b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.954448 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-2430e891-a1ce-4cc9-9c33-3ea51734ae94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434b712c217a47b79ecd966efbd8428b'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:40.961905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1323/2630] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:40 2022] DELETE /identity/v3/users/434b712c217a47b79ecd966efbd8428b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:40.965214 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3875bd47e8f748b68e985bb095c08339, user_id=96b0028d546b441495e246bc434a4c7e, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:40.966260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.966540 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.966808 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:40.967937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696c013-966a-4149-8ca6-5abcbebaa65f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:40.974608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:40.982317 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1308/2631] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:40 2022] PUT /identity/v3/projects/3875bd47e8f748b68e985bb095c08339/users/96b0028d546b441495e246bc434a4c7e/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:14:40.986983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:40.988548 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:40.992570 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:40.992876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:40.993131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.019712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] MFA Rules not processed for user `96b0028d546b441495e246bc434a4c7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:41.020931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.049321 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1324/2632] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:41.059862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.061279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.086620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1309/2633] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:41.093215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.094604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.114362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6961d-b0ae-46fe-8579-11b52ddcfba9 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.121963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.122796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.123224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.123654 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.124687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.128088 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.138492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd996b-1bad-4ea9-8b8f-4a1a74141f83 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.144003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.144933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.145382 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.145848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.158502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.160987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.169102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1310/2634] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:41.175160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.175805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1325/2635] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:41.225602 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:41.248130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116ffcd-ec26-4ae5-83df-a7259f5a8deb admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.253134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.253786 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.254090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.254336 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.254969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.256920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.279561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1311/2636] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:41.283798 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.294505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcc71f-55fb-4f9e-a65f-679da625c85b service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.299539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.300125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.300395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.300657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.301288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.303507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.313583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1326/2637] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:41.317947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.331941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd5350-1072-4ed3-8ccf-50100cb40462 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.336974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.337779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.338124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.338485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] PATH_INFO: `/v3/projects/3e86665f6560435488c1fc8b7cae45be/users/2761942f1c2b41bb99675e03dae856d7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.354195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e86665f6560435488c1fc8b7cae45be, user_id=2761942f1c2b41bb99675e03dae856d7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.356932 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.370717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1312/2638] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:41 2022] PUT /identity/v3/projects/3e86665f6560435488c1fc8b7cae45be/users/2761942f1c2b41bb99675e03dae856d7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:41.375031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.385012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387dba54-3448-40b3-9734-cc9792ef21cf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.448447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.449087 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.449328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.449627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.450288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.451893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.462183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.468595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1327/2639] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:41.468805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.469644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.480091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e827397-db29-4c7e-a0a6-1663ce44c04f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.484893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.485608 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.485887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.486156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] PATH_INFO: `/v3/projects/3e86665f6560435488c1fc8b7cae45be/users/2761942f1c2b41bb99675e03dae856d7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.488274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e86665f6560435488c1fc8b7cae45be, user_id=2761942f1c2b41bb99675e03dae856d7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.489350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e79d-c56e-4eff-a76f-7ee3270ff620 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.490960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6eff66-bbea-46e9-a47e-0cf0af653997 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.496419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.497261 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.497744 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.498187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] PATH_INFO: `/v3/projects/b4c690be95794f15913662bf3c709407` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.499647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c690be95794f15913662bf3c709407)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.501592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.504759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1328/2640] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:41 2022] PUT /identity/v3/projects/3e86665f6560435488c1fc8b7cae45be/users/2761942f1c2b41bb99675e03dae856d7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:41.514420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.514702 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.515054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.550417 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] MFA Rules not processed for user `2761942f1c2b41bb99675e03dae856d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:41.551768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.555007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c690be95794f15913662bf3c709407', 'request_id': 'req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:41.560717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c690be95794f15913662bf3c709407', 'request_id': 'req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:41.561070 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:41.570008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c690be95794f15913662bf3c709407', 'request_id': 'req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:41.581971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] The token cache is being invalidate because project b4c690be95794f15913662bf3c709407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:41.582268 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:41.582982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-18884ba7-6956-4e3a-b9c9-0d7d1dc5fc88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:41.584501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1313/2641] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:41 2022] DELETE /identity/v3/projects/b4c690be95794f15913662bf3c709407 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:41.589351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4fc1-e129-4209-8dd7-df962ff7bcde None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.589664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4fc1-e129-4209-8dd7-df962ff7bcde None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.589940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c4fc1-e129-4209-8dd7-df962ff7bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.609260 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-297c4fc1-e129-4209-8dd7-df962ff7bcde None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:41.610547 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-297c4fc1-e129-4209-8dd7-df962ff7bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.626297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1329/2642] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:41.637510 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.638936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.670457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1314/2643] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:41.699526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3f1b6-2a8d-4952-b969-ea23575aea21 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.703381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.704014 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.704282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.704546 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.714850 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.716368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.727684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1330/2644] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:14:41.739399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.739683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.739955 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.761319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:41.762576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.771437 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1315/2645] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:41.775903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.780154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.781690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.787778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef76b3-03b7-4642-a099-a72238f5fa68 service cinder] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.792972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.793605 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.793886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.794155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.794303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fb381-fa5e-4239-8c8a-3c0ab500a501 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.794804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:41.795103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.796818 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.798735 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.799374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.799648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.799915 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.801952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:41.809405 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1331/2646] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:41.814013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.823439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:41.824282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ef801-bc2a-40e8-a848-1fb23de7bafb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.829656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.830164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.830423 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.830684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.831305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.833393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.862602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1332/2647] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:41.864762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.866318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.866782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.875312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1316/2648] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:14:41.929734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b130-9483-4309-89c2-03028c2cccb9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.934936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.935532 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.935803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.936063 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.936684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.938696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.961639 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1333/2649] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:41 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:41.966232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:41.977236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf215-6fb5-4f4d-8995-024ef62e41a2 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:41.983657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:41.984353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:41.984596 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:41.984893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:41.985701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:41.987330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:41.998651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1317/2650] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:42.002971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:42.008940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:42.014880 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edfcb1-04ce-488c-9967-ab38a99e4d9b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:42.021676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:42.022619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:42.022894 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:42.023216 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:42.023874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:42.025638 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6211cba-827a-44b6-9e44-c34f33d0b084 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:42.036284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1334/2651] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:42.040753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:42.051881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1445a1-147e-4a83-a583-935356940ed2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:42.057203 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:42.057922 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:42.058199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:42.058468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] PATH_INFO: `/v3/projects/49988974815d4fc28b3f7bf347a9b8ec/users/b52ce281858448859aa9f3f67fea9152/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:42.080224 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49988974815d4fc28b3f7bf347a9b8ec, user_id=b52ce281858448859aa9f3f67fea9152, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:42.083125 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:42.097023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1318/2652] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:42 2022] PUT /identity/v3/projects/49988974815d4fc28b3f7bf347a9b8ec/users/b52ce281858448859aa9f3f67fea9152/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:42.107261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:42.107562 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:42.107843 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:42.139313 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] MFA Rules not processed for user `b52ce281858448859aa9f3f67fea9152`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:42.140529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:42.209710 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1335/2653] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:42.222190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:42.277868 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0edaf-8bf8-429d-a165-23b5560858d5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:42.282001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:42.282601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:42.282872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:42.283128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:42.291555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:42.292963 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5809ab-4cdc-405f-b3be-7851c5ce27ac service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:42.301833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1319/2654] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:45.251603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:45.253269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:45.319438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a3ba6-4863-4275-9b12-cc88f5340ec6 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:45.324319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:45.324937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:45.325210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:45.325480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] PATH_INFO: `/v3/users/96b0028d546b441495e246bc434a4c7e` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:45.326657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b0028d546b441495e246bc434a4c7e)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:45.328597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:45.390161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b0028d546b441495e246bc434a4c7e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.400663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b0028d546b441495e246bc434a4c7e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.415147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b0028d546b441495e246bc434a4c7e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.424925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b0028d546b441495e246bc434a4c7e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.425803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b0028d546b441495e246bc434a4c7e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.428946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:45.434081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1336/2655] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:45 2022] DELETE /identity/v3/users/96b0028d546b441495e246bc434a4c7e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:45.439484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:45.440002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:45.440002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:45.466711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:45.468559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:45.531814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1320/2656] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:45.541703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:45.543279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:45.601457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075e7b-6a57-43ff-b23d-a14b2690a218 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:45.605470 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:45.606094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:45.606366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:45.606627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:45.617897 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:45.619488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:45.630258 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1337/2657] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:45.912451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:45.914297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:45.927129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab74ff-6bea-4c07-baa6-7e1ca0f71284 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:45.932157 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:45.932805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:45.933143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:45.933340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] PATH_INFO: `/v3/projects/3875bd47e8f748b68e985bb095c08339` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:45.934554 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3875bd47e8f748b68e985bb095c08339)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:45.936465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:45.962477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:45.987212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3875bd47e8f748b68e985bb095c08339', 'request_id': 'req-ccfa95f5-8887-4474-907c-74dc5f09d027'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:45.992724 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3875bd47e8f748b68e985bb095c08339', 'request_id': 'req-ccfa95f5-8887-4474-907c-74dc5f09d027'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.002579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3875bd47e8f748b68e985bb095c08339', 'request_id': 'req-ccfa95f5-8887-4474-907c-74dc5f09d027'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.014196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] The token cache is being invalidate because project 3875bd47e8f748b68e985bb095c08339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:46.014479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.015186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.016701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1321/2658] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:45 2022] DELETE /identity/v3/projects/3875bd47e8f748b68e985bb095c08339 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:46.020840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.031000 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f82d-0d64-4cd8-8e13-1238129bb982 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.089584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.090228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.090499 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.090751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] PATH_INFO: `/v3/users/2761942f1c2b41bb99675e03dae856d7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.091914 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2761942f1c2b41bb99675e03dae856d7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.093934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.140941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2761942f1c2b41bb99675e03dae856d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.147868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2761942f1c2b41bb99675e03dae856d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.160618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2761942f1c2b41bb99675e03dae856d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.170446 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2761942f1c2b41bb99675e03dae856d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.171490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-e1f05b11-5871-419d-a466-9005125c2dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2761942f1c2b41bb99675e03dae856d7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.177657 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1338/2659] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:46 2022] DELETE /identity/v3/users/2761942f1c2b41bb99675e03dae856d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:46.451351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.452945 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:46.466563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa95f5-8887-4474-907c-74dc5f09d027 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.526153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.526788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.527060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.527351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] PATH_INFO: `/v3/projects/3e86665f6560435488c1fc8b7cae45be` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.528471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e86665f6560435488c1fc8b7cae45be)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.530389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.581248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e86665f6560435488c1fc8b7cae45be', 'request_id': 'req-02ff1446-cb5b-4edc-add6-42fa6c1942b5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.587737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e86665f6560435488c1fc8b7cae45be', 'request_id': 'req-02ff1446-cb5b-4edc-add6-42fa6c1942b5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.598037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e86665f6560435488c1fc8b7cae45be', 'request_id': 'req-02ff1446-cb5b-4edc-add6-42fa6c1942b5'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.609306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] The token cache is being invalidate because project 3e86665f6560435488c1fc8b7cae45be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:46.609617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.610263 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-02ff1446-cb5b-4edc-add6-42fa6c1942b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:46.611758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1322/2660] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:46 2022] DELETE /identity/v3/projects/3e86665f6560435488c1fc8b7cae45be => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:46.616958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.617443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.617557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.637173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:46.638402 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:46.655165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:46.700332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1339/2661] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:46.709422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.709741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.710012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.730782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:46.731973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:46.740447 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1323/2662] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:46.744716 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.755223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67d4c4-e306-4376-ad3f-ef080fd488ed None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.761080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.761729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.762005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.762273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.762953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:46.763275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.764925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.777091 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1340/2663] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:46.782071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.795755 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffa01d-738c-4b1f-aab3-ae8d1cba4ddd None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.803502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.804223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.804599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.805473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.806960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.809525 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.840643 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1324/2664] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:46 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:46.846985 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.857877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee24d6-761d-4d22-ac76-91e537e0ded6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.916143 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.916738 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.917010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.917272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.917934 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.919876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.942632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1341/2665] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:46 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:46.947054 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.957563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743b801-727d-4d5f-b871-72fcf032f31e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.963493 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.964092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.964371 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.964634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.965271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:46.966902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:46.977485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1325/2666] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:46.981428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:46.991603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2df8-5aad-4376-b126-7830c38a8542 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:46.996480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:46.997110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:46.997908 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:46.998185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:46.998841 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.000526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fcee0-6b2c-43bd-b5ce-8aaefc9b6797 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.011550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1342/2667] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:47.017700 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.026040 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:47.031961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ae8e7-eb82-4e2d-ba57-c4d7cffdb623 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.038815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.039556 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.039914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.040190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] PATH_INFO: `/v3/projects/720c5758c54f4c2e94144aa0c52e236b/users/3ae418f8a2f54d73a27b5abf612b0568/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.056167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=720c5758c54f4c2e94144aa0c52e236b, user_id=3ae418f8a2f54d73a27b5abf612b0568, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.058882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.074480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1326/2668] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:47 2022] PUT /identity/v3/projects/720c5758c54f4c2e94144aa0c52e236b/users/3ae418f8a2f54d73a27b5abf612b0568/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:47.084609 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.084891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.085159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.115658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] MFA Rules not processed for user `3ae418f8a2f54d73a27b5abf612b0568`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:47.116912 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:47.191018 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1343/2669] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:47.195646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.259650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f591c-05ff-405f-84ea-31c41abc6aed admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.264816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.265469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.265811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.266242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.266853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.268737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.300143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1327/2670] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:47 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:47.304739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.315694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d66f7-e0a9-48ea-b3e7-f9389c3aa37f None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.329012 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:47.376411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.377021 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.377295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.377572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.378204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.380138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.403493 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1344/2671] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:47 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:47.408198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.419543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a601e-4aeb-4b52-929a-96790def0729 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.424855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.425460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.425768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.426039 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.426681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.428618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.439435 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1328/2672] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:47.443963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.455056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d0d7b-1947-4807-b38d-d0cab13c45e1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.460974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.461665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.461939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.462202 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] PATH_INFO: `/v3/projects/231a0db2391744b7806f4b6451a53702/users/ccadc69cfcc94caab99e9983b8f3612b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.477763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a0db2391744b7806f4b6451a53702, user_id=ccadc69cfcc94caab99e9983b8f3612b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.480415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.496457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1345/2673] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:47 2022] PUT /identity/v3/projects/231a0db2391744b7806f4b6451a53702/users/ccadc69cfcc94caab99e9983b8f3612b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:47.500559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.510845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a1a4b-073d-4711-a87e-1a43dca431e5 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.572323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.572856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.573134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.573408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.574081 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.575675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7c850-8adc-48f3-8f88-f6648efb9ac2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.585563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1329/2674] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:47.589946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.601149 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ec1ba-957d-498b-aa53-f0bb20184ce6 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.606341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.606988 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.607260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.607526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] PATH_INFO: `/v3/projects/231a0db2391744b7806f4b6451a53702/users/ccadc69cfcc94caab99e9983b8f3612b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.609549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231a0db2391744b7806f4b6451a53702, user_id=ccadc69cfcc94caab99e9983b8f3612b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.612203 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.622785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1346/2675] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:47 2022] PUT /identity/v3/projects/231a0db2391744b7806f4b6451a53702/users/ccadc69cfcc94caab99e9983b8f3612b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:47.633068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.633354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.633658 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.661230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] MFA Rules not processed for user `ccadc69cfcc94caab99e9983b8f3612b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:47.662455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:47.700687 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:47.735172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1330/2676] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:47.746065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:47.747933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:47.807439 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47810c13-5918-4986-be29-8212e619a990 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:47.811653 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:47.812281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:47.812558 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:47.812823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:47.822262 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:47.823768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6023e-04f3-4651-8885-2bc4ca191536 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:47.833728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1347/2677] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:48.410128 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:48.476300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c230fe-84d9-4854-b38d-2f630497890d None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:48.481252 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:48.481916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:48.482236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:48.482524 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] PATH_INFO: `/v3/users/3ae418f8a2f54d73a27b5abf612b0568` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:48.483877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ae418f8a2f54d73a27b5abf612b0568)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:48.485864 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:48.532212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ae418f8a2f54d73a27b5abf612b0568'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:48.532887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ae418f8a2f54d73a27b5abf612b0568'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:48.533462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ae418f8a2f54d73a27b5abf612b0568'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:48.545220 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ae418f8a2f54d73a27b5abf612b0568'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:48.554171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ae418f8a2f54d73a27b5abf612b0568'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:48.569262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1331/2678] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:48 2022] DELETE /identity/v3/users/3ae418f8a2f54d73a27b5abf612b0568 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:48.574128 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:48.574412 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:48.574680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:48.594166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:48.595317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:48.658324 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1348/2679] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:48.670333 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:48.672850 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:48.744725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2030f-f2cc-494f-88c3-0bd925851f9e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:48.748928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:48.749579 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:48.749889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:48.750182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:48.757363 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:48.759623 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:48.761298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:48.771625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1332/2680] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:49.042555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.044151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:49.056917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047cb70-cbf9-4588-b180-2961831a32be None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:49.062144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:49.062823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.063099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.063370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] PATH_INFO: `/v3/projects/720c5758c54f4c2e94144aa0c52e236b` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.064526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720c5758c54f4c2e94144aa0c52e236b)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:49.066515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:49.075270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:49.117851 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720c5758c54f4c2e94144aa0c52e236b', 'request_id': 'req-488a1933-ec74-4217-8b25-00740e811d80'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.123261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720c5758c54f4c2e94144aa0c52e236b', 'request_id': 'req-488a1933-ec74-4217-8b25-00740e811d80'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.129746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720c5758c54f4c2e94144aa0c52e236b', 'request_id': 'req-488a1933-ec74-4217-8b25-00740e811d80'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.144810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] The token cache is being invalidate because project 720c5758c54f4c2e94144aa0c52e236b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:49.145129 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.145900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.147574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1349/2681] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:49 2022] DELETE /identity/v3/projects/720c5758c54f4c2e94144aa0c52e236b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:49.151846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.163601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946fd79-4272-41aa-a505-3bf1d333e28e service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:49.222347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:49.222973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.223249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.223517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] PATH_INFO: `/v3/users/ccadc69cfcc94caab99e9983b8f3612b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.224742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccadc69cfcc94caab99e9983b8f3612b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:49.226691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:49.278094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccadc69cfcc94caab99e9983b8f3612b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.278714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccadc69cfcc94caab99e9983b8f3612b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.279319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccadc69cfcc94caab99e9983b8f3612b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.291306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccadc69cfcc94caab99e9983b8f3612b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.298829 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-8bfd5a4c-4258-47b3-ab32-d99d2f7948f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccadc69cfcc94caab99e9983b8f3612b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.314624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1333/2682] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:49 2022] DELETE /identity/v3/users/ccadc69cfcc94caab99e9983b8f3612b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:49.591571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.593261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:49.608667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1933-ec74-4217-8b25-00740e811d80 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:49.673794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:49.674476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.674685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.674949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] PATH_INFO: `/v3/projects/231a0db2391744b7806f4b6451a53702` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.676057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231a0db2391744b7806f4b6451a53702)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:49.678019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:49.724962 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231a0db2391744b7806f4b6451a53702', 'request_id': 'req-5406911e-b048-41a2-b14f-301a2e7066d5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.730243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231a0db2391744b7806f4b6451a53702', 'request_id': 'req-5406911e-b048-41a2-b14f-301a2e7066d5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.736198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231a0db2391744b7806f4b6451a53702', 'request_id': 'req-5406911e-b048-41a2-b14f-301a2e7066d5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.750492 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] The token cache is being invalidate because project 231a0db2391744b7806f4b6451a53702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:49.750782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.751551 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-5406911e-b048-41a2-b14f-301a2e7066d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:49.753349 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1350/2683] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:49 2022] DELETE /identity/v3/projects/231a0db2391744b7806f4b6451a53702 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:49.758207 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.758691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.758936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.779018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:49.780310 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:49.843196 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1334/2684] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:49.851277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.851555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.851807 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.872071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:49.873227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:49.881495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1351/2685] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:49.885873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.896360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a5247-dcd3-4578-9511-9f85ab72d3b2 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:49.901764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:49.902373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.902682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.902950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.903598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:49.903897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:49.905504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:49.918617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1335/2686] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:49.923139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.935422 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9508f1-8b22-440f-9ef9-038540f91120 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:49.942403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:49.943130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:49.943410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:49.943674 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:49.944488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:49.946495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:49.970331 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:49.979535 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1352/2687] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:49 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:49.983721 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:49.988937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:49.995099 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec5884-24cf-4aa4-9f77-9eef925bfe71 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.054768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.055378 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.055657 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.055930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.056542 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.058504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.081662 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1336/2688] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:49 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:50.086053 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.097075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c8bbd-e7ad-4572-8179-7576e0d0fb48 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.102427 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.103017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.103290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.103555 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.104213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.105929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.115910 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1353/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:50.120177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.130588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a9a82-bab6-49a3-91e2-c778cc058ad4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.136111 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.136712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.136988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.137270 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.137940 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.139557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66d09-616e-45e4-8d81-29fbd3f2e583 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.150514 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1337/2690] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:50.154744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.165327 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba3ab-cea6-46ab-87df-a2771814fc8e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.171272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.171947 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.172226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.172497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] PATH_INFO: `/v3/projects/3c29baab0023403e875fb63e9f0deac8/users/1b4d6c5cedc44edca406ffe789f22499/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.188536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c29baab0023403e875fb63e9f0deac8, user_id=1b4d6c5cedc44edca406ffe789f22499, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.191284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.206559 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1354/2691] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:50 2022] PUT /identity/v3/projects/3c29baab0023403e875fb63e9f0deac8/users/1b4d6c5cedc44edca406ffe789f22499/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:14:50.217032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.217315 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.217597 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.245752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] MFA Rules not processed for user `1b4d6c5cedc44edca406ffe789f22499`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:50.246965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.312614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1338/2692] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:50.318026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.390261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683f03e-ea2b-4734-9fb6-bf550e202e26 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.395983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.396618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.396890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.397123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.397795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.399707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.416183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.417983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.429893 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1355/2693] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:50 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:50.432445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714afdc-a07d-4b39-9d45-836f89a23f93 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.434045 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.490255 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.490899 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.491176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.491446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] PATH_INFO: `/v3/users/b52ce281858448859aa9f3f67fea9152` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.492611 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52ce281858448859aa9f3f67fea9152)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.494598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.498440 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506139c3-2164-4d28-8ea7-3ec65b1d974c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.503335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.503969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.504237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.504498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.505165 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.507153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.507694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:50.531938 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1356/2694] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:50 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:50.536163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.546476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c0c6-40ed-454e-82dc-18613f0e07e4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.551379 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.551975 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.552202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52ce281858448859aa9f3f67fea9152'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:50.552311 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.552626 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.552930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52ce281858448859aa9f3f67fea9152'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:50.553656 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.553947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52ce281858448859aa9f3f67fea9152'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:50.555253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.571420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52ce281858448859aa9f3f67fea9152'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:50.571628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1357/2695] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:50.573370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.574488 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-84657644-f8e9-4c78-85c6-b14a0bc9b44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52ce281858448859aa9f3f67fea9152'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:50.575287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.588254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1339/2696] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:50 2022] DELETE /identity/v3/users/b52ce281858448859aa9f3f67fea9152 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:50.588573 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e96313-d5c5-4b03-8651-658eccf8e4bb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.592626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.592911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.593186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.611613 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:50.611810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:50.612816 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.646287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.646875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.647146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.647411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] PATH_INFO: `/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37/users/df82fd7453e9420993e29fcd2a6751a7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.661762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2fb96020b84f7c86fc91a82a6fbf37, user_id=df82fd7453e9420993e29fcd2a6751a7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.664424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.671998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1340/2697] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:50.679456 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1358/2698] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:50 2022] PUT /identity/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37/users/df82fd7453e9420993e29fcd2a6751a7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:50.682377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.683139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.683719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.737959 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aae269-a320-496c-a2ea-c22388f41e5f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.744514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abca86e-1369-4ac1-b64e-7c74475d8479 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.749467 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.750095 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.750369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.750635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.751345 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.752946 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.762828 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1359/2699] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:50.766876 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.776977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5541ba-1ec6-428a-b1bc-55b321cca437 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:50.781628 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:50.783288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.784265 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.784607 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.784933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] PATH_INFO: `/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37/users/df82fd7453e9420993e29fcd2a6751a7/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.787189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2fb96020b84f7c86fc91a82a6fbf37, user_id=df82fd7453e9420993e29fcd2a6751a7, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.788110 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:50.788753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.789256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.789758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.790114 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae777b2-38ec-410c-ac8e-3d90d90ec148 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.799877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:50.801196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1360/2700] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:50 2022] PUT /identity/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37/users/df82fd7453e9420993e29fcd2a6751a7/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:50.801332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:50.811093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1341/2701] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:50.875681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:50.876065 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:50.876387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:50.906953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] MFA Rules not processed for user `df82fd7453e9420993e29fcd2a6751a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:50.908124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:50.984793 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1361/2702] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:50.996511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:50.997948 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.056185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56570-e6ae-4076-a6c8-2fe8c3f638be service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:51.096206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:51.097762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.113715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:51.114186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:51.114485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:51.114764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:51.120729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:51.125326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:51.126974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:51.139372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1342/2703] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:51.170652 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa9f7-57ad-4c29-bbaf-eb43004841a4 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:51.174052 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:51.174773 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:51.175108 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:51.175434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] PATH_INFO: `/v3/projects/49988974815d4fc28b3f7bf347a9b8ec` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:51.176692 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49988974815d4fc28b3f7bf347a9b8ec)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:51.179125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:51.220383 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:51.229415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c04a2f-c328-4f8b-877b-dcf2bc5f07bb service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:51.234195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49988974815d4fc28b3f7bf347a9b8ec', 'request_id': 'req-dd2af757-7076-4892-9bcb-1362e57e6679'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.239870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49988974815d4fc28b3f7bf347a9b8ec', 'request_id': 'req-dd2af757-7076-4892-9bcb-1362e57e6679'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.246315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49988974815d4fc28b3f7bf347a9b8ec', 'request_id': 'req-dd2af757-7076-4892-9bcb-1362e57e6679'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.261226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] The token cache is being invalidate because project 49988974815d4fc28b3f7bf347a9b8ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:51.261634 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.262560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.264556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1362/2704] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:51 2022] DELETE /identity/v3/projects/49988974815d4fc28b3f7bf347a9b8ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:51.279796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:51.280432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:51.280712 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:51.280987 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:51.282989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.349505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:51.350995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8da9fe-2c83-49bd-b3da-75562968b17d service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:51.361582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1343/2705] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:14:51.657096 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:51.658584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.731802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af757-7076-4892-9bcb-1362e57e6679 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:51.737826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:51.738450 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:51.738719 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:51.738980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] PATH_INFO: `/v3/users/1b4d6c5cedc44edca406ffe789f22499` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:51.740146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4d6c5cedc44edca406ffe789f22499)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:51.742117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:51.791760 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4d6c5cedc44edca406ffe789f22499'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.793774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:51.799918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4d6c5cedc44edca406ffe789f22499'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.812981 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4d6c5cedc44edca406ffe789f22499'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.823217 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4d6c5cedc44edca406ffe789f22499'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.823954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4d6c5cedc44edca406ffe789f22499'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:51.830600 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1363/2706] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:51 2022] DELETE /identity/v3/users/1b4d6c5cedc44edca406ffe789f22499 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:51.835231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:51.835742 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:51.835840 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:51.843256 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:51.857461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:51.858718 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.923391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1344/2707] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:51.934389 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:51.936009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:51.996058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8def05b-c4ea-4bb5-b3f8-dd50f167bbf1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:52.000717 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:52.001368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:52.001699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:52.001960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:52.011915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:52.013370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:52.022825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1364/2708] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:52.312131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:52.313873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:52.325885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4df84-a47e-4549-8bd4-97ffa18cd83b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:52.331077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:52.331710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:52.332432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:52.332432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] PATH_INFO: `/v3/projects/3c29baab0023403e875fb63e9f0deac8` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:52.333338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c29baab0023403e875fb63e9f0deac8)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:52.335513 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:52.383692 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c29baab0023403e875fb63e9f0deac8', 'request_id': 'req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.388939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c29baab0023403e875fb63e9f0deac8', 'request_id': 'req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.398139 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c29baab0023403e875fb63e9f0deac8', 'request_id': 'req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.409648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] The token cache is being invalidate because project 3c29baab0023403e875fb63e9f0deac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:52.409937 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.410573 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.412068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1345/2709] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:52 2022] DELETE /identity/v3/projects/3c29baab0023403e875fb63e9f0deac8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:14:52.416019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:52.425855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b1fdd-ea53-4aa8-8f4d-9f47838172c5 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:52.483848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:52.484475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:52.484744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:52.485019 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] PATH_INFO: `/v3/users/df82fd7453e9420993e29fcd2a6751a7` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:52.486205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df82fd7453e9420993e29fcd2a6751a7)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:52.488117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:52.536933 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df82fd7453e9420993e29fcd2a6751a7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.546503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df82fd7453e9420993e29fcd2a6751a7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.558997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:52.560502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df82fd7453e9420993e29fcd2a6751a7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.573194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df82fd7453e9420993e29fcd2a6751a7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.573907 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-fc3372b4-5d9c-4a3a-8f1b-8f569a4b2d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df82fd7453e9420993e29fcd2a6751a7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.580353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1365/2710] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:14:52 2022] DELETE /identity/v3/users/df82fd7453e9420993e29fcd2a6751a7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:52.856516 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:52.858494 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:52.876793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6d13c-22e2-4a9d-ab32-9085b27f87c0 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:52.936072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:52.936705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:52.936979 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:52.937249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] PATH_INFO: `/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:52.938382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2fb96020b84f7c86fc91a82a6fbf37)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:52.940328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:52.972734 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:52.989466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2fb96020b84f7c86fc91a82a6fbf37', 'request_id': 'req-d8e1b134-4290-44f5-ba20-d202d1126bb6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:52.995173 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2fb96020b84f7c86fc91a82a6fbf37', 'request_id': 'req-d8e1b134-4290-44f5-ba20-d202d1126bb6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:53.003555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2fb96020b84f7c86fc91a82a6fbf37', 'request_id': 'req-d8e1b134-4290-44f5-ba20-d202d1126bb6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:53.015703 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] The token cache is being invalidate because project aa2fb96020b84f7c86fc91a82a6fbf37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:14:53.015989 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:53.016722 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d8e1b134-4290-44f5-ba20-d202d1126bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:14:53.018349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1346/2711] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:14:52 2022] DELETE /identity/v3/projects/aa2fb96020b84f7c86fc91a82a6fbf37 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:53.024680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.025174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.025174 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.046849 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:53.048009 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:53.110553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1366/2712] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:14:53.118817 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.119100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.119394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.140361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:53.141607 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:53.150879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1347/2713] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:53.155275 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.165385 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ae394-fb8b-464d-8219-df38259271f8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.170494 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.171119 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.171388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.171647 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.172308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:14:53.172602 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.174399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.186776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1367/2714] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:14:53.191172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.200887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd894f-d36e-4bfd-b164-e041587407be None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.206886 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.207487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.207761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.208031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.208663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.210624 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.238361 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1348/2715] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:53 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:53.242665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.252722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938efcb-1e38-4b5e-a7c8-032d1a63c948 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.321210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.321765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.322048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.322315 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.322936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.324892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.348380 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1368/2716] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:53 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:53.352599 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.363274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dcc23-1ecf-4a9c-99ec-26b8eba8b3f4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.368885 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.369480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.369788 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.370053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.370690 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.372431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.382834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1349/2717] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:53.386746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.397111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1dae1-5e9c-450c-8c78-392ceda1bce5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.402135 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.402730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.403005 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.403274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.403918 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.406241 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cfd58-8a51-4c73-9488-76028a8e620d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.416632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1369/2718] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:14:53.420711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.432047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6ffb9-f5c8-48f3-ac9d-544625b417d2 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.438245 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.438974 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.439250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.439571 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] PATH_INFO: `/v3/projects/458eee750aa741c493481ee785761172/users/112f1de9bcdc4b8fb3d402190d4f0751/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.456976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] RBAC: Authorizing `identity:create_grant(project_id=458eee750aa741c493481ee785761172, user_id=112f1de9bcdc4b8fb3d402190d4f0751, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.459636 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.475474 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1350/2719] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:53 2022] PUT /identity/v3/projects/458eee750aa741c493481ee785761172/users/112f1de9bcdc4b8fb3d402190d4f0751/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:53.481711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c24c3-4b74-4b55-9eb5-041cc0355dbe None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.482084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c24c3-4b74-4b55-9eb5-041cc0355dbe None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.482478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c24c3-4b74-4b55-9eb5-041cc0355dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.508742 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-b34c24c3-4b74-4b55-9eb5-041cc0355dbe None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:53.509937 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b34c24c3-4b74-4b55-9eb5-041cc0355dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:53.561401 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:53.576779 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1370/2720] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:53.588806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:53.591920 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:53.653757 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c017713-db22-4e9f-b3fa-ad13f2ca823f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:53.658300 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:53.658933 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:53.659215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:53.659478 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:53.673470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:53.675559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:53.690301 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1351/2721] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:14:57.676372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:57.676852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:57.676935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:57.718531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] MFA Rules not processed for user `112f1de9bcdc4b8fb3d402190d4f0751`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:14:57.719769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:14:57.788614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1371/2722] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:14:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:14:57.793101 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:57.803160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090971-2a6c-4dbf-988b-7b9c6e85976e service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:57.808168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:57.808779 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:57.809057 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:57.809328 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:57.809998 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:57.811912 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:57.840402 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1352/2723] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:14:57 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:14:57.844563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:57.855903 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2b549-26c4-4f4a-a4c3-237e997d80e8 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:57.917930 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:57.918785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:57.919188 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:57.919543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:57.920276 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:57.922554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:57.950909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1372/2724] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:14:57 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:14:57.957126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:57.968082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1f4d-3c5c-4158-9f10-9e9e109fc362 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:57.972918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:57.973645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:57.973929 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:57.974419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:57.975100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:57.976699 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:57.988358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1353/2725] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:57.992564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:58.003209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fd4d6-8199-4997-be80-e5c7b79d053f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:58.007891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:58.008580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:58.008855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:58.009124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] PATH_INFO: `/v3/projects/9d1f911a05ac422487d9b8fb15732f25/users/66e6af15e4144be4ad1560e7216b1aa3/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:58.024982 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d1f911a05ac422487d9b8fb15732f25, user_id=66e6af15e4144be4ad1560e7216b1aa3, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:58.027694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:58.042699 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1373/2726] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:57 2022] PUT /identity/v3/projects/9d1f911a05ac422487d9b8fb15732f25/users/66e6af15e4144be4ad1560e7216b1aa3/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:14:58.046914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:58.057787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9478dc-c57b-48ff-96ae-693ed3ec54f9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:58.077185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:14:58.126561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:58.127264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:58.127537 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:58.127805 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:58.128449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:58.130087 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79dc07c-a8a3-4982-ac6c-7a959977eab3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:58.141441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1354/2727] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:14:58.145820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:14:58.156287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b49ef-9cb6-4afe-ad28-18c839e1692b admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:14:58.161205 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:14:58.161892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:14:58.162164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:14:58.162431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] PATH_INFO: `/v3/projects/9d1f911a05ac422487d9b8fb15732f25/users/66e6af15e4144be4ad1560e7216b1aa3/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:14:58.164341 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d1f911a05ac422487d9b8fb15732f25, user_id=66e6af15e4144be4ad1560e7216b1aa3, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:14:58.167040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:14:58.177940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1374/2728] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:14:58 2022] PUT /identity/v3/projects/9d1f911a05ac422487d9b8fb15732f25/users/66e6af15e4144be4ad1560e7216b1aa3/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:15:02.461142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6be45-1d43-4686-8c32-24645ad0569b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:02.461809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6be45-1d43-4686-8c32-24645ad0569b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:02.461809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6be45-1d43-4686-8c32-24645ad0569b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:02.497566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-b9f6be45-1d43-4686-8c32-24645ad0569b None None] MFA Rules not processed for user `66e6af15e4144be4ad1560e7216b1aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:15:02.498867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b9f6be45-1d43-4686-8c32-24645ad0569b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:02.584748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1355/2729] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:15:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:15:02.620299 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:02.622281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:02.673178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:15:02.708042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba1c7b-171f-4397-8dd0-ed8481164f85 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:02.713758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:02.714377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:02.714650 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:02.714929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:02.774490 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:02.776139 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:02.785890 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1375/2730] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:04.082592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc5f3a-a925-4fa8-adc0-ba861c0e8257 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:04.083246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc5f3a-a925-4fa8-adc0-ba861c0e8257 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:04.083246 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc5f3a-a925-4fa8-adc0-ba861c0e8257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:04.104204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bdfc5f3a-a925-4fa8-adc0-ba861c0e8257 None None] MFA Rules not processed for user `112f1de9bcdc4b8fb3d402190d4f0751`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:15:04.105468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bdfc5f3a-a925-4fa8-adc0-ba861c0e8257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:04.113621 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1356/2731] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:15:12.781794 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=39 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:15:14.097726 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=56 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:15:20.084062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:20.096734 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ed63f-0ace-45c8-8feb-19ab6593df82 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:20.106050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:20.106802 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:20.107192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:20.107554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:20.109560 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:20.125039 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:20.127464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:20.140081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1376/2732] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:23.357277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:23.357931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:23.357931 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:23.377470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] MFA Rules not processed for user `31a49b84974b480985d0b002fc740e95`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:15:23.378702 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:23.437307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1357/2733] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:15:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:15:23.757711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:23.759263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:23.818971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454d45f-d746-48ab-a717-d0820f1ae3a5 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:23.822704 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:23.823986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:23.824261 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:23.824519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:23.826375 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:23.837959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:23.839443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:23.849592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1377/2734] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:24.308425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:24.310098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:24.389669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f6a92-0bed-424d-9819-40d9e12a90ba None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:24.394641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:24.395254 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:24.395536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:24.395806 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:24.397521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:24.409160 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:24.410856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:24.421053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1358/2735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:31.159167 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:31.160954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:31.177486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799142a-a461-44c3-b33a-93244e988c9c service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:31.183604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:31.184209 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:31.184485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:31.184750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:31.186579 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:31.197172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:31.198801 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:31.207737 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1378/2736] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:34.052512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:34.066569 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfabd49-4af8-4a6f-844f-dc88b968ac76 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:34.072815 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:34.073436 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:34.073745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:34.074020 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:34.075960 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:15:34.088967 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:34.090589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:34.101323 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1359/2737] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:41.203771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=38 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:15:44.097183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=63 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:15:44.809809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:15:44.820227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b52cbe-4b61-4aa0-a5ca-7d27e1a73eb1 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:15:44.825069 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:15:44.825697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:15:44.825969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:15:44.826229 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:15:44.837433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:15:44.838906 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e30940-f11f-479a-b2e5-dd81c02e2649 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:15:44.848676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1379/2738] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:15:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:15:54.844640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:07.261028 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:07.262521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:07.277170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5e834-8733-4acf-8ed8-cf33e1d1c2a4 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:07.282670 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:07.283276 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:07.283548 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:07.283813 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:07.285869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:07.298391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:07.299980 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:07.311030 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1360/2739] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:12.029026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:12.029599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:12.029599 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:12.049451 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:12.050753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:12.108614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1380/2740] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:17:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:12.162572 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:12.164095 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:12.245853 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568007e-4e5c-499b-8734-f82adeffc65e service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:12.252566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:12.253668 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:12.254120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:12.254583 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:12.267957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:12.269745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f158e-fb9a-42c8-9a8a-346cfd9f1c7c service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:12.278834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1361/2741] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:17:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 17 20:17:22.101455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=25 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:22.274745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=38 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:25.578771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:25.588714 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c894ee-91e0-429b-ae8e-5fdeeb74897e None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:25.593173 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:25.593811 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:25.594084 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:25.594350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:25.596316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:25.606293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:25.607726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148fa479-53b2-410c-bd04-aa4644390aaf service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:25.615751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1381/2742] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:25.744204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:25.744764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:25.744764 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:25.746285 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1362/2743] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:17:25.752497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d403-3e81-45ca-9258-c105fc4d330b None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:25.752786 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d403-3e81-45ca-9258-c105fc4d330b None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:25.753056 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2d403-3e81-45ca-9258-c105fc4d330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:25.773681 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-49f2d403-3e81-45ca-9258-c105fc4d330b None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:25.774931 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-49f2d403-3e81-45ca-9258-c105fc4d330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:25.785762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1382/2744] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:17:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:25.797097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:25.799152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:25.886349 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24043d4-e916-4b2f-8da5-8a80f9af1d54 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:25.890114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:25.890716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:25.890984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:25.891244 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:25.901129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:25.902564 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769c1a9-ec5f-47fb-9c95-bc5206ba5410 service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:25.912694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1363/2745] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:25.991574 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30fbd4-1f8c-4313-8e0f-5c86cf37bb0d None None] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:25.991803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30fbd4-1f8c-4313-8e0f-5c86cf37bb0d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:25.992102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30fbd4-1f8c-4313-8e0f-5c86cf37bb0d None None] PATH_INFO: `/` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:25.993630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1383/2746] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:17:26.000736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a964a31-9277-4e01-85de-0178d366e499 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:26.000995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a964a31-9277-4e01-85de-0178d366e499 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:26.001264 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a964a31-9277-4e01-85de-0178d366e499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:26.040716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7a964a31-9277-4e01-85de-0178d366e499 None None] MFA Rules not processed for user `4eae163ceee54be9a10f7475ab87392e`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:26.041975 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7a964a31-9277-4e01-85de-0178d366e499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:26.049684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1364/2747] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:17:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:26.062013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30fbd4-1f8c-4313-8e0f-5c86cf37bb0d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:26.074965 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30fbd4-1f8c-4313-8e0f-5c86cf37bb0d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:26.079549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:26.080150 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:26.080421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:26.080687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:26.088279 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:26.089777 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e651c41-3df1-4133-b9a9-3278cf5a1005 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:26.097651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1384/2748] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:26.186855 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689080cb-d7f5-491e-bb61-27c07ded802c None None] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:26.187186 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689080cb-d7f5-491e-bb61-27c07ded802c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:26.187565 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689080cb-d7f5-491e-bb61-27c07ded802c None None] PATH_INFO: `/` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:26.189306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1365/2749] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 17 20:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 17 20:17:26.197721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ee20f-4bcd-417b-a2b6-904c1d4492e0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:26.198006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ee20f-4bcd-417b-a2b6-904c1d4492e0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:26.198273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ee20f-4bcd-417b-a2b6-904c1d4492e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:26.219350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0d6ee20f-4bcd-417b-a2b6-904c1d4492e0 None None] MFA Rules not processed for user `1244993f093f4ad0920e3a42725a2308`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:26.220568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0d6ee20f-4bcd-417b-a2b6-904c1d4492e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:26.231769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1385/2750] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 17 20:17:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:26.245164 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689080cb-d7f5-491e-bb61-27c07ded802c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:26.257675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689080cb-d7f5-491e-bb61-27c07ded802c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:26.261767 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:26.262360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:26.262634 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:26.262893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:26.274480 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:26.275914 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:26.286598 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1366/2751] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:28.297273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:28.297791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:28.297791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:28.317977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:28.319172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:28.327792 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1386/2752] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:17:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:28.343059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:28.358459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb2c06-dc0a-494d-b973-5da57ea05aa9 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:28.364632 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:28.365234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:28.365505 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:28.365791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:28.374161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:28.375663 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7cb85-ab29-43fe-9042-e40d173c06de service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:28.383719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1367/2753] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:17:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:17:38.312744 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=21 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:38.380659 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=56 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:46.175957 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:46.190974 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718c020-e5c5-4241-b2b7-923f9234464d None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:46.198296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:46.199115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:46.199452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:46.199758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:46.203006 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:46.218121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:46.219569 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:46.230545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1387/2754] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:46.423641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:46.424145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:46.424145 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:46.447430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:17:46.449177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:46.460694 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1368/2755] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:17:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:17:46.473185 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:46.485288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393a53-ccc0-4ddf-ab60-387eddf88242 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:46.491596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:46.492227 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:46.492518 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:46.492755 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:46.501661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:46.503269 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:46.511508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1388/2756] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:17:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:17:47.421728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:17:47.432445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94ff6-aabf-4424-b000-aefab355727e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:17:47.436601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:17:47.437199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:17:47.437471 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:17:47.437768 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:17:47.439923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:17:47.454984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:17:47.456408 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:17:47.467555 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1369/2757] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:17:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:17:56.508282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=12 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:17:57.462832 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=9 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:15.953529 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:15.963098 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2463ae-736d-46c6-b6ab-a549292426ff service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:15.967081 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:15.967745 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:15.968085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:15.968415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:15.970256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:15.980788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:15.982429 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:15.991291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1389/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:16.085181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:16.094799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b8fe1-72ee-4c9a-b795-8642c712828d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:16.099691 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:16.100222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:16.100536 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:16.100798 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:16.102725 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:16.116870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:16.118334 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:16.129311 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1370/2759] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:25.987571 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:26.124479 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:50.088113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:50.098340 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac87d9-b00b-42c2-8b5b-ec345706c117 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:50.102987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:50.103586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:50.103928 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:50.104297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:50.106613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:50.175318 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:50.177317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:50.192544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1390/2760] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:53.254308 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:53.266581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa375e-384a-4cc4-a1e2-a055505c7c9b service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:53.273074 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:53.273996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.274432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.274876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.276871 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:53.291075 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:53.293826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:53.312179 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1371/2761] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:53.324852 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:53.326486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:53.346810 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbd4b-39ab-46d5-923b-f1f040902094 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:53.354246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:53.355164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.355485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.355783 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] PATH_INFO: `/v3/users/31a49b84974b480985d0b002fc740e95` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.357500 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31a49b84974b480985d0b002fc740e95)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:53.359628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:53.414256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31a49b84974b480985d0b002fc740e95'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.421646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31a49b84974b480985d0b002fc740e95'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.434434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31a49b84974b480985d0b002fc740e95'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.444463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31a49b84974b480985d0b002fc740e95'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.445097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4a4d233-3272-48de-9b36-9d5af3339093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31a49b84974b480985d0b002fc740e95'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.451452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1391/2762] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:18:53 2022] DELETE /identity/v3/users/31a49b84974b480985d0b002fc740e95 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:53.635563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:53.647158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa0bb1d-a9e2-4ef8-99a1-f33644d97390 service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:53.704450 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:53.705080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.705354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.705652 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] PATH_INFO: `/v3/projects/129d0816f8ef4b4f8c8d626b49a03679` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.706785 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=129d0816f8ef4b4f8c8d626b49a03679)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:53.709936 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:53.759530 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129d0816f8ef4b4f8c8d626b49a03679', 'request_id': 'req-834fda86-641f-4222-9f67-cda7155717bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.766013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129d0816f8ef4b4f8c8d626b49a03679', 'request_id': 'req-834fda86-641f-4222-9f67-cda7155717bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.776340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129d0816f8ef4b4f8c8d626b49a03679', 'request_id': 'req-834fda86-641f-4222-9f67-cda7155717bf'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.789771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] The token cache is being invalidate because project 129d0816f8ef4b4f8c8d626b49a03679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:18:53.790105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.790820 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-834fda86-641f-4222-9f67-cda7155717bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:53.792473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1372/2763] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:18:53 2022] DELETE /identity/v3/projects/129d0816f8ef4b4f8c8d626b49a03679 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:53.797507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.797868 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.798140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.818739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:53.819997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:53.883351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1392/2764] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:53.891793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.892080 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.892346 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.912492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:53.913650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:53.922172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1373/2765] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:53.926648 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:53.937259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947212d7-b592-4591-a860-d9edf7f9a193 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:53.943195 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:53.943819 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.944083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.944377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.945015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81533) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:18:53.945310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:53.946945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:53.948396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:53.963944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1393/2766] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:18:53.965172 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:53.976350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d088a12-8c8d-4bf9-8abd-7f3951401d72 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:53.981976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:53.982582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:53.982856 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:53.983120 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:53.983745 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:53.985727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.018682 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1374/2767] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:18:53 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:54.023099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.033454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9039049-c66d-449e-abde-9a47446e74b9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.115746 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.116364 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.116637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.116887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.117505 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.119491 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.143310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1394/2768] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:18:54 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:54.148568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.159266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda8f6a-b961-40ee-b8c9-8b65c26af9bc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.164615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.165154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.165428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.165732 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.166374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.167961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.178461 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1375/2769] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:54.182900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.193481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b399b-591f-46be-aaf0-07da78604390 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.198515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.199111 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.199390 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.199659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.200300 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.201953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78289a-ba4f-49e6-90ec-deaa5c2712cd admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.212730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1395/2770] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:54.217008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.227153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1817f32-47a8-428b-8908-bdb197af8e05 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.232113 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.232754 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.233032 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.233303 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] PATH_INFO: `/v3/projects/d3b41edcf90344549eb9a86ec325516a/users/fee12ae2d5b3448882802404b7f9cb41/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.248052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3b41edcf90344549eb9a86ec325516a, user_id=fee12ae2d5b3448882802404b7f9cb41, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.250802 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.266625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1376/2771] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:54 2022] PUT /identity/v3/projects/d3b41edcf90344549eb9a86ec325516a/users/fee12ae2d5b3448882802404b7f9cb41/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:54.278960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.279248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.279515 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.309112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] MFA Rules not processed for user `fee12ae2d5b3448882802404b7f9cb41`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:54.310383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:54.386622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1396/2772] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:54.391391 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.406037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:54.455809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f55b1-f0a8-4d3b-944f-f4d012835211 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.460898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.461511 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.461822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.462077 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.462711 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.464618 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.492003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1377/2773] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:18:54 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:54.496977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.511369 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e03d5-c5c9-4669-84bc-13daa158e287 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.570594 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.571190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.571853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.571853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] PATH_INFO: `/v3/users` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.572354 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.574361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.583661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:54.599210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1397/2774] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:18:54 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:54.603347 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.613796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29cdd2-f1df-4ce2-8d3d-8fccbfc77961 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.618462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.619092 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.619374 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.619641 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.620330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.621952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.632545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1378/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:54.636855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.647267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40909-5490-4951-aaad-d21bbe3c629f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.653348 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.654010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.654282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.654544 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] PATH_INFO: `/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce/users/7deab7eb4aea437e81d0d248f5855f26/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.670610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb4ff60fb4d442d9c1a97e3458380ce, user_id=7deab7eb4aea437e81d0d248f5855f26, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.673501 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.689317 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1398/2776] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:54 2022] PUT /identity/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce/users/7deab7eb4aea437e81d0d248f5855f26/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:18:54.693727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.703970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670060d-1d23-4605-8aee-6c85475f0e46 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.764683 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.765208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.765485 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.765787 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.766462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.768037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48a102-7d58-4674-8e8e-d980f0557c3f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.778079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1379/2777] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:54.782536 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.793527 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0716a-2a08-477f-b1d7-ea9befa147b3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:54.798830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:54.799506 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.799770 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.800040 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] PATH_INFO: `/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce/users/7deab7eb4aea437e81d0d248f5855f26/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.802146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb4ff60fb4d442d9c1a97e3458380ce, user_id=7deab7eb4aea437e81d0d248f5855f26, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:54.805204 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:54.815750 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1399/2778] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:54 2022] PUT /identity/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce/users/7deab7eb4aea437e81d0d248f5855f26/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:54.826234 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:54.826521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:54.826791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:54.856071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] MFA Rules not processed for user `7deab7eb4aea437e81d0d248f5855f26`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:54.857653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:54.933822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1380/2779] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:54.945441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:54.947254 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:55.008214 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab183ff-4af9-4774-9e15-56635c5079af admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:55.012050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:55.012665 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:55.012939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:55.013208 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:55.023048 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:55.024539 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc9735-560d-46eb-b953-4129e1795caa service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:55.035029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1400/2780] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:55.485822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:55.496106 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:55.550799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909275f6-259c-4154-956e-f8b0d62bb282 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:55.556135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:55.556758 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:55.557060 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:55.557326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] PATH_INFO: `/v3/users/fee12ae2d5b3448882802404b7f9cb41` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:55.558499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee12ae2d5b3448882802404b7f9cb41)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:55.560412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:55.610630 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee12ae2d5b3448882802404b7f9cb41'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:55.611297 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee12ae2d5b3448882802404b7f9cb41'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:55.611924 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee12ae2d5b3448882802404b7f9cb41'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:55.623560 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee12ae2d5b3448882802404b7f9cb41'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:55.630177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee12ae2d5b3448882802404b7f9cb41'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:55.648843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1381/2781] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:18:55 2022] DELETE /identity/v3/users/fee12ae2d5b3448882802404b7f9cb41 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:55.653622 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:55.654140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:55.654140 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:55.674921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:55.676142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:55.866664 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1401/2782] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:55.876747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:55.878232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:55.935887 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a510c5-b299-4997-be0c-85fd8b92f9cc admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:55.939918 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:55.940519 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:55.940796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:55.941069 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:55.950981 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:55.952422 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:55.961994 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1382/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:18:56.243488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:56.245027 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:56.258372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e4036-23d0-4a5e-bb18-5acf6a0d53eb None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:56.264292 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:56.264923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:56.265193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:56.265452 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] PATH_INFO: `/v3/projects/d3b41edcf90344549eb9a86ec325516a` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:56.266768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3b41edcf90344549eb9a86ec325516a)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:56.268853 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:56.320370 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3b41edcf90344549eb9a86ec325516a', 'request_id': 'req-1fb1c852-a478-4030-bf69-f2a959cc0ce7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.322459 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:56.327032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3b41edcf90344549eb9a86ec325516a', 'request_id': 'req-1fb1c852-a478-4030-bf69-f2a959cc0ce7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.334867 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3b41edcf90344549eb9a86ec325516a', 'request_id': 'req-1fb1c852-a478-4030-bf69-f2a959cc0ce7'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.349187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] The token cache is being invalidate because project d3b41edcf90344549eb9a86ec325516a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:18:56.349473 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.350206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.351677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1402/2784] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:18:56 2022] DELETE /identity/v3/projects/d3b41edcf90344549eb9a86ec325516a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:56.355791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:56.366027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1709-eb71-49e8-8a94-9b51d3a1a288 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:56.396592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:56.427034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:56.427661 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:56.427938 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:56.428206 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] PATH_INFO: `/v3/users/7deab7eb4aea437e81d0d248f5855f26` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:56.429733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7deab7eb4aea437e81d0d248f5855f26)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:56.431645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:56.481358 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7deab7eb4aea437e81d0d248f5855f26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.482103 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7deab7eb4aea437e81d0d248f5855f26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.482766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7deab7eb4aea437e81d0d248f5855f26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.494793 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7deab7eb4aea437e81d0d248f5855f26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.502093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-b3a99ed4-41b2-4c0c-8e61-412114e82af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7deab7eb4aea437e81d0d248f5855f26'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.517776 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1383/2785] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:18:56 2022] DELETE /identity/v3/users/7deab7eb4aea437e81d0d248f5855f26 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:56.795909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:56.797646 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:56.812557 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1c852-a478-4030-bf69-f2a959cc0ce7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:56.869553 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:56.870194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:56.870463 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:56.870726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] PATH_INFO: `/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:56.871838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb4ff60fb4d442d9c1a97e3458380ce)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:56.873767 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:56.921788 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb4ff60fb4d442d9c1a97e3458380ce', 'request_id': 'req-eb07dd05-e4b3-4306-b59a-8b75600bf843'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.927366 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb4ff60fb4d442d9c1a97e3458380ce', 'request_id': 'req-eb07dd05-e4b3-4306-b59a-8b75600bf843'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.933992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb4ff60fb4d442d9c1a97e3458380ce', 'request_id': 'req-eb07dd05-e4b3-4306-b59a-8b75600bf843'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.949372 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] The token cache is being invalidate because project abb4ff60fb4d442d9c1a97e3458380ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:18:56.949705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.950361 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-eb07dd05-e4b3-4306-b59a-8b75600bf843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:18:56.951883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1403/2786] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:18:56 2022] DELETE /identity/v3/projects/abb4ff60fb4d442d9c1a97e3458380ce => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:56.957761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:56.958049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:56.958321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:56.979135 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:56.980337 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:57.045218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1384/2787] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 17 20:18:57.053480 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.053796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.054407 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.074285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:57.075468 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:57.083878 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1404/2788] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:57.088526 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.098988 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8b81f-aa9a-4709-8a8c-d242077658ab None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.104302 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.104892 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.105170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.105431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.106130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:18:57.106429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.108010 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.120090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1385/2789] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:18:57.124959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.138549 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794ceb6-7e0e-4110-a184-69913c0aa1d2 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.146211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.147074 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.147431 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.147795 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.148672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.150643 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.183457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1405/2790] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/projects => generated 401 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:57.187680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.198341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeda639-247a-4630-abf6-bec99d6801e4 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.232085 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:57.257122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.257748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.258022 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.258287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.258910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.260819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.284215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1386/2791] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:18:57.288483 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.299035 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492dc5f1-93d4-4551-9eaf-dc03135f9368 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.303739 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.304683 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.305031 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.305295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.305958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.307543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.317508 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1406/2792] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:57.321705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.331795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d043d-30ba-4c03-85cc-9e9405d1be48 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.336459 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.337058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.337338 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.337647 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.338304 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.339901 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7527f2-f375-4ed6-b043-7c8744e9c133 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.350633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1387/2793] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:57.354954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.365408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed65f6-3047-4186-99ba-118c4d73fa5c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.370220 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.370857 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.371122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.371383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] PATH_INFO: `/v3/projects/f6fd2b4304074293a253cfa2ee15c80d/users/553f624100f242e3b154482c7da2b6d9/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.386525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fd2b4304074293a253cfa2ee15c80d, user_id=553f624100f242e3b154482c7da2b6d9, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.389142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.405116 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1407/2794] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:57 2022] PUT /identity/v3/projects/f6fd2b4304074293a253cfa2ee15c80d/users/553f624100f242e3b154482c7da2b6d9/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:57.416923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.417211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.417483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.446414 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] MFA Rules not processed for user `553f624100f242e3b154482c7da2b6d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:57.447566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:57.521509 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1388/2795] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:57.526164 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.536533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:57.594998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406a1eb-b696-4b35-8839-b5d30e13f1e7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.600082 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.600706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.600987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.601250 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.601901 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.604066 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.637017 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1408/2796] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:57.641332 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.653082 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eede013-0cf4-4ac9-8ffe-6c61dc1bb1b0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.714949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.715470 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.715746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.716014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.716626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.718568 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.742558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1389/2797] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:18:57 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:18:57.746832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.757309 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0b533-bc2a-4b5a-8487-b7973ad59dc2 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.762865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.763464 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.763736 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.763998 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.764641 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.766334 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.776764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1409/2798] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:57.781425 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.796272 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9c90f-91c7-4629-b35b-1494e7b90966 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.803416 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.804672 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.805031 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.805350 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] PATH_INFO: `/v3/projects/2adec7822a7a4f92ad9c4627504316e5/users/b3b5eda1baff4a4b91f24f7401a512c9/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.821991 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adec7822a7a4f92ad9c4627504316e5, user_id=b3b5eda1baff4a4b91f24f7401a512c9, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.824751 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.841710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1390/2799] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:57 2022] PUT /identity/v3/projects/2adec7822a7a4f92ad9c4627504316e5/users/b3b5eda1baff4a4b91f24f7401a512c9/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:57.845864 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.859666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07fda1-92de-45a9-aead-687e935a0708 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.935286 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.935892 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.936163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.936426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.937061 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.938691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252df2c-9c11-49e5-8ec4-a71978151cf6 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.949270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1410/2800] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:18:57.954982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:57.966153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9160-dcba-43b5-83cd-6e5665190319 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:57.971384 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:57.972036 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:57.972321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:57.972590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] PATH_INFO: `/v3/projects/2adec7822a7a4f92ad9c4627504316e5/users/b3b5eda1baff4a4b91f24f7401a512c9/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:57.974704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adec7822a7a4f92ad9c4627504316e5, user_id=b3b5eda1baff4a4b91f24f7401a512c9, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:57.977318 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:57.983221 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:57.989675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1391/2801] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:18:57 2022] PUT /identity/v3/projects/2adec7822a7a4f92ad9c4627504316e5/users/b3b5eda1baff4a4b91f24f7401a512c9/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:18:58.004820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:58.005163 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:58.005339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:58.036552 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] MFA Rules not processed for user `b3b5eda1baff4a4b91f24f7401a512c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:18:58.037817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:58.119175 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1411/2802] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:18:58 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:18:58.130400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:58.132502 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:18:58.192559 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff147-2eae-45fb-b783-a4e5a1492398 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:58.196773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:58.197373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:58.197675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:58.197942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:58.207601 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:58.209046 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:58.220553 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1392/2803] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:58 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:18:58.307125 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:18:58.311913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:18:58.318156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339c462-2358-49a7-97b7-c06c808605f5 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:18:58.322384 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:18:58.322989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:18:58.323257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:18:58.323523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:18:58.380516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:18:58.382002 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d186e51-25fc-45ff-bca1-2c7ac255a94e service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:18:58.390722 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1412/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:18:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:02.540518 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:02.635321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b434a-9292-40bc-a653-f12f1dc5c038 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:02.640161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:02.640803 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:02.641079 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:02.641354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] PATH_INFO: `/v3/users/553f624100f242e3b154482c7da2b6d9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:02.642532 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] RBAC: Authorizing `identity:delete_user(user_id=553f624100f242e3b154482c7da2b6d9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:02.644469 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:02.694399 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553f624100f242e3b154482c7da2b6d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:02.695073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553f624100f242e3b154482c7da2b6d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:02.695679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553f624100f242e3b154482c7da2b6d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:02.709181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553f624100f242e3b154482c7da2b6d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:02.716521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553f624100f242e3b154482c7da2b6d9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:02.732953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1393/2805] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:19:02 2022] DELETE /identity/v3/users/553f624100f242e3b154482c7da2b6d9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:02.737731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:02.738253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:02.738355 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:02.758223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:02.759554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:02.823804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1413/2806] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:02.834034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:02.835577 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:02.864674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:02.904608 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8a236-b98f-4ae4-982e-94127d71d86d admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:02.908603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:02.909222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:02.909498 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:02.909879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:02.920774 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:02.922306 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:02.932748 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1394/2807] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:03.218525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:03.220113 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:03.232940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfd082-e14a-4794-abe1-305088f40bf0 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:03.238604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:03.239253 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:03.239526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:03.239774 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] PATH_INFO: `/v3/projects/f6fd2b4304074293a253cfa2ee15c80d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:03.241041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fd2b4304074293a253cfa2ee15c80d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:03.243001 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:03.307144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fd2b4304074293a253cfa2ee15c80d', 'request_id': 'req-1a0723dc-8907-445f-a042-bd3d0c04a341'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.313145 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fd2b4304074293a253cfa2ee15c80d', 'request_id': 'req-1a0723dc-8907-445f-a042-bd3d0c04a341'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.319531 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fd2b4304074293a253cfa2ee15c80d', 'request_id': 'req-1a0723dc-8907-445f-a042-bd3d0c04a341'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.332630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] The token cache is being invalidate because project f6fd2b4304074293a253cfa2ee15c80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:19:03.332900 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.333676 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.335201 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1414/2808] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:19:03 2022] DELETE /identity/v3/projects/f6fd2b4304074293a253cfa2ee15c80d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:03.339484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:03.349070 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa3a04-b41c-46ad-b6c7-79ff72f2d410 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:03.411964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:03.412492 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:03.412770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:03.413041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] PATH_INFO: `/v3/users/b3b5eda1baff4a4b91f24f7401a512c9` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:03.414201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b5eda1baff4a4b91f24f7401a512c9)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:03.416211 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:03.475664 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b5eda1baff4a4b91f24f7401a512c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.476325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b5eda1baff4a4b91f24f7401a512c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.476962 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b5eda1baff4a4b91f24f7401a512c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.489451 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b5eda1baff4a4b91f24f7401a512c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.496360 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-d4a65acd-e84c-4656-b169-9ce01dc22ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b5eda1baff4a4b91f24f7401a512c9'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.511413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1395/2809] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:19:03 2022] DELETE /identity/v3/users/b3b5eda1baff4a4b91f24f7401a512c9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:19:03.816191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:03.817762 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:03.832273 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0723dc-8907-445f-a042-bd3d0c04a341 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:03.860137 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:03.904359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:03.905010 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:03.905285 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:03.905603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] PATH_INFO: `/v3/projects/2adec7822a7a4f92ad9c4627504316e5` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:03.906817 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adec7822a7a4f92ad9c4627504316e5)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:03.908733 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:03.961408 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adec7822a7a4f92ad9c4627504316e5', 'request_id': 'req-62fb1e05-ce23-45d8-a858-740877065329'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.967929 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adec7822a7a4f92ad9c4627504316e5', 'request_id': 'req-62fb1e05-ce23-45d8-a858-740877065329'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.974628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adec7822a7a4f92ad9c4627504316e5', 'request_id': 'req-62fb1e05-ce23-45d8-a858-740877065329'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.992680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] The token cache is being invalidate because project 2adec7822a7a4f92ad9c4627504316e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:19:03.992954 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.993684 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-62fb1e05-ce23-45d8-a858-740877065329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:03.995211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1415/2810] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:19:03 2022] DELETE /identity/v3/projects/2adec7822a7a4f92ad9c4627504316e5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:04.000377 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.000888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.000888 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.021517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:04.022782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:04.062953 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:04.087730 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1396/2811] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:04.096944 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.097350 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.097735 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.120729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:04.122130 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:04.132156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1416/2812] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:04.162909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.173826 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a36fbc-2313-4d7c-97d5-e0681e536edc None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.178966 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.179923 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.180197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.180473 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.181131 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:19:04.181426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.183053 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.195972 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1397/2813] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:19:04.200680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.215689 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f53f03-bad8-48d7-9ede-19a915029786 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.221122 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.221838 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.222144 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.222403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.223041 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.225194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.260190 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1417/2814] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:04.264527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.275126 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d2ce8-058b-462f-a368-2475fe4bcb6f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.348457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.349076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.349351 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.349653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.350287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.352258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.376185 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1398/2815] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:04.380554 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.391381 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef43c39-6a84-4043-84b5-8635c4b7b0e9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.396356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.396958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.397226 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.397486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.398162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.400004 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.411245 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1418/2816] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:04.416012 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.426655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475a3e9-7efe-49c8-90e1-c8ebd3df9534 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.433184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.433916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.434247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.434557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.435274 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.437027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19208b4-467a-451a-9922-98068d0191b6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.450183 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1399/2817] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:19:04.454690 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.465580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4251-b257-473f-b25c-98f625ecf0f5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.471987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.472591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.472869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.473142 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] PATH_INFO: `/v3/projects/0668da494d254777b8a245ac8b885cc0/users/a2681d2912194b99aac4e50315163a8f/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.493567 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0668da494d254777b8a245ac8b885cc0, user_id=a2681d2912194b99aac4e50315163a8f, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.496306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.516199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1419/2818] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:04 2022] PUT /identity/v3/projects/0668da494d254777b8a245ac8b885cc0/users/a2681d2912194b99aac4e50315163a8f/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:04.527016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.527295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.527550 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.558736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] MFA Rules not processed for user `a2681d2912194b99aac4e50315163a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:04.559983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:04.630766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1400/2819] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:04.635545 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.691503 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:04.708240 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e710c-8034-4fa7-8264-057b9ae01b97 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.713870 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.714484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.714757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.715023 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.715666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.717592 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.752231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1420/2820] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:04.756710 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.768468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1a20e-cb6d-4862-b54d-6854d080896e None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.809008 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:04.838296 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.838916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.839197 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.839466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.840102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.842056 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.866782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1401/2821] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:19:04 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:04.872297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.886778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b2d74-0639-493e-9658-694cc4e15a99 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.892691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.893310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.893613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.893875 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.894526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.896176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.907860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1421/2822] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:04.912168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.927617 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b567-ba7d-41fd-ab41-2791ec880045 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.934763 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.935741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.936226 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.936705 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.937903 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:04.940984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696683e-4464-45f7-8363-62de0bbbd416 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:04.960228 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1402/2823] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:04.964678 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:04.975215 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33839c-f415-4ecb-ae7a-18d4ea3830f3 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:04.979899 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:04.980565 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:04.980834 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:04.981099 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] PATH_INFO: `/v3/projects/ea4b1e726afd45538df224ad7e9764d6/users/ffb593c89f8844f3bbf76cd4888256fa/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:04.998042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4b1e726afd45538df224ad7e9764d6, user_id=ffb593c89f8844f3bbf76cd4888256fa, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.000753 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.016771 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1422/2824] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:04 2022] PUT /identity/v3/projects/ea4b1e726afd45538df224ad7e9764d6/users/ffb593c89f8844f3bbf76cd4888256fa/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:05.027327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.027893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.027893 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.059004 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] MFA Rules not processed for user `ffb593c89f8844f3bbf76cd4888256fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:05.060230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:05.132486 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1403/2825] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:05.137711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.207751 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa80ef-66c2-430c-8239-6ad50b51e032 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.212858 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.213476 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.213925 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.214360 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.215386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.218873 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.259050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1423/2826] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:19:05 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:05.263882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.276170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0fbcd-2e30-4c02-8b82-c520251b7cba None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.339151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.339766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.340058 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.340329 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.340957 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.342939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.366846 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1404/2827] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:19:05 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:05.371395 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.382403 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b70d33-ac96-4b62-b570-0d29d3d877a9 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.388089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.388711 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.388989 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.389256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.389958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.391586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.403296 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1424/2828] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:05.408415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.419295 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21bd44-a5e8-4b1e-a01b-dc5de69cfe6c admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.424073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.424733 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.425013 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.425288 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] PATH_INFO: `/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572/users/9d5742ea134d4ae993d3ea27e1a2586b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.442995 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcc7a5e43e84ff1bde7a80c629be572, user_id=9d5742ea134d4ae993d3ea27e1a2586b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.445707 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.462501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1405/2829] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:05 2022] PUT /identity/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572/users/9d5742ea134d4ae993d3ea27e1a2586b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:05.466895 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.477741 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d229e-b5fa-48d0-af8a-373f6b6e160d admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.549953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.550485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.550758 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.551026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.551709 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.553329 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade186fd-8e3f-4410-86fa-04ea9eb0a03b admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.564949 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1425/2830] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:05.570911 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.583889 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130715-049a-412f-a8c6-d4a12e8a82a3 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.589468 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.590379 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.590750 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.591165 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] PATH_INFO: `/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572/users/9d5742ea134d4ae993d3ea27e1a2586b/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.593698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bcc7a5e43e84ff1bde7a80c629be572, user_id=9d5742ea134d4ae993d3ea27e1a2586b, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.596576 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.604709 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:05.614429 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1406/2831] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:05 2022] PUT /identity/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572/users/9d5742ea134d4ae993d3ea27e1a2586b/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:19:05.627776 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.628067 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.628371 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.652977 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:05.663424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] MFA Rules not processed for user `9d5742ea134d4ae993d3ea27e1a2586b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:05.664677 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:05.747481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1426/2832] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:05.759455 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:05.761201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:05.825984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1381d7-e859-4fed-80e5-52b1865a8be9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:05.829588 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:05.830222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:05.830507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:05.830770 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:05.901143 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:05.903242 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:05.914589 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1407/2833] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:06.892003 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:06.893865 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:06.964416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e38b6e-cea3-4c17-a58b-0e7bea5b838a None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:06.968572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:06.969294 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:06.969706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:06.970079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:06.972155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:06.983358 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:06.985958 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:06.999564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1427/2834] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:07.573902 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:07.575781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:07.588094 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52191b7-7d69-4e48-ad82-6b7251ab8752 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:07.592490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:07.593184 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:07.593466 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:07.593766 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:07.671105 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:07.672897 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:07.688688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1408/2835] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:08.362706 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:08.374339 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68e12c-ea2c-4acc-9558-72826f76a261 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:08.378322 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:08.378940 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:08.379213 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:08.379479 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:08.391399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:08.394177 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24c27a-47b6-49fc-aba7-99a78c735874 service cinder] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:08.410855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1428/2836] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:12.626562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:12.691752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=100 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:12.708014 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb11bf-a6e7-4ad6-829f-4e0c1f8ab700 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:12.713162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:12.713876 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:12.714158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:12.714434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] PATH_INFO: `/v3/users/a2681d2912194b99aac4e50315163a8f` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:12.715756 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2681d2912194b99aac4e50315163a8f)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:12.717769 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:12.780822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2681d2912194b99aac4e50315163a8f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:12.781863 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2681d2912194b99aac4e50315163a8f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:12.782828 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2681d2912194b99aac4e50315163a8f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:12.803446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2681d2912194b99aac4e50315163a8f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:12.815167 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2681d2912194b99aac4e50315163a8f'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:12.832620 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1409/2837] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:19:12 2022] DELETE /identity/v3/users/a2681d2912194b99aac4e50315163a8f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:12.839478 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:12.840159 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:12.840288 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:12.871015 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:12.873131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:12.977399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1429/2838] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:12.987627 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:12.989279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:13.066462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23074f-e86c-4c36-97e0-d0626fcd6c45 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:13.071799 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:13.072747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:13.073177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:13.073645 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:13.088326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:13.089818 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:13.100715 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1410/2839] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:13.440696 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:13.442416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:13.470328 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1d1d7-c592-4dd5-a43b-38e8fdd180c9 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:13.478507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:13.479623 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:13.480162 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:13.481192 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] PATH_INFO: `/v3/projects/0668da494d254777b8a245ac8b885cc0` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:13.482057 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0668da494d254777b8a245ac8b885cc0)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:13.484310 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:13.512290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:13.552488 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0668da494d254777b8a245ac8b885cc0', 'request_id': 'req-8b223e54-57e5-4498-b2ac-f18e69e2546e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.559281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0668da494d254777b8a245ac8b885cc0', 'request_id': 'req-8b223e54-57e5-4498-b2ac-f18e69e2546e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.569987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0668da494d254777b8a245ac8b885cc0', 'request_id': 'req-8b223e54-57e5-4498-b2ac-f18e69e2546e'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.589123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] The token cache is being invalidate because project 0668da494d254777b8a245ac8b885cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:19:13.589481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.590118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.591644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1430/2840] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:19:13 2022] DELETE /identity/v3/projects/0668da494d254777b8a245ac8b885cc0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:19:13.595965 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:13.609882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d8482-29ed-42cf-af30-49ee00941e9d service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:13.697651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:13.698507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:13.698795 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:13.699073 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] PATH_INFO: `/v3/users/ffb593c89f8844f3bbf76cd4888256fa` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:13.700307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb593c89f8844f3bbf76cd4888256fa)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:13.702357 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:13.722150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:13.771671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb593c89f8844f3bbf76cd4888256fa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.776973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb593c89f8844f3bbf76cd4888256fa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.776973 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb593c89f8844f3bbf76cd4888256fa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.792086 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb593c89f8844f3bbf76cd4888256fa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.803098 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb593c89f8844f3bbf76cd4888256fa'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:13.822984 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1411/2841] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:19:13 2022] DELETE /identity/v3/users/ffb593c89f8844f3bbf76cd4888256fa => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:14.175743 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:14.177628 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:14.195347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223e54-57e5-4498-b2ac-f18e69e2546e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:14.272013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:14.272654 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:14.272913 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:14.273181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] PATH_INFO: `/v3/projects/ea4b1e726afd45538df224ad7e9764d6` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:14.274528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4b1e726afd45538df224ad7e9764d6)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:14.276528 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:14.335765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4b1e726afd45538df224ad7e9764d6', 'request_id': 'req-ce34eab7-3161-4e7e-90f5-a77e42545c12'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.342862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4b1e726afd45538df224ad7e9764d6', 'request_id': 'req-ce34eab7-3161-4e7e-90f5-a77e42545c12'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.351260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4b1e726afd45538df224ad7e9764d6', 'request_id': 'req-ce34eab7-3161-4e7e-90f5-a77e42545c12'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.368172 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] The token cache is being invalidate because project ea4b1e726afd45538df224ad7e9764d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:19:14.368498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.369155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.370731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1431/2842] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:19:14 2022] DELETE /identity/v3/projects/ea4b1e726afd45538df224ad7e9764d6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:14.374964 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:14.376653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:14.393581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533387c-0ef0-4e43-953a-659750ca8108 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:14.480506 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:14.481064 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:14.481342 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:14.481655 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] PATH_INFO: `/v3/users/9d5742ea134d4ae993d3ea27e1a2586b` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:14.482842 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5742ea134d4ae993d3ea27e1a2586b)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:14.485052 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:14.560171 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5742ea134d4ae993d3ea27e1a2586b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.561158 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5742ea134d4ae993d3ea27e1a2586b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.562059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5742ea134d4ae993d3ea27e1a2586b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.580746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5742ea134d4ae993d3ea27e1a2586b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.588452 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-9b57ea13-c3cc-4b85-89a0-0db067e97fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5742ea134d4ae993d3ea27e1a2586b'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:14.609869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1412/2843] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:19:14 2022] DELETE /identity/v3/users/9d5742ea134d4ae993d3ea27e1a2586b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:14.972525 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:14.974941 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:14.996433 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34eab7-3161-4e7e-90f5-a77e42545c12 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.056118 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:15.080487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.081199 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.081694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.082206 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] PATH_INFO: `/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.083916 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bcc7a5e43e84ff1bde7a80c629be572)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.086161 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.148806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bcc7a5e43e84ff1bde7a80c629be572', 'request_id': 'req-b4202669-5414-4835-942e-e290593aa8f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:15.162685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bcc7a5e43e84ff1bde7a80c629be572', 'request_id': 'req-b4202669-5414-4835-942e-e290593aa8f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:15.170485 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bcc7a5e43e84ff1bde7a80c629be572', 'request_id': 'req-b4202669-5414-4835-942e-e290593aa8f4'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:15.188406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] The token cache is being invalidate because project 8bcc7a5e43e84ff1bde7a80c629be572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:19:15.189191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:15.189428 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b4202669-5414-4835-942e-e290593aa8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:19:15.191421 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1432/2844] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:19:14 2022] DELETE /identity/v3/projects/8bcc7a5e43e84ff1bde7a80c629be572 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:15.197262 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.197916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.197916 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.219068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:15.221015 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:15.229249 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:15.301421 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1413/2845] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:15.310484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.310484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.310705 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.343651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:15.344935 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:15.355644 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1433/2846] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:15.361123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.374292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99b2b6-7da9-4e02-8938-8c5e8d122ab6 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.382219 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.383218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.383651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.384041 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.384928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:19:15.385359 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.387218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.403956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1414/2847] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:19:15.408765 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.426597 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aeb09b-c2da-44a6-8872-b4f9c1290524 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.433766 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.434416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.434685 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.434951 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.435618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.437658 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.479187 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1434/2848] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:19:15 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:15.485154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.497068 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28470a-7449-4d37-9e5e-26db0d3759cf admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.562625 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.563200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.563404 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.563684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.564362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.566401 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.592182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1415/2849] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:19:15 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:15.596085 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.607821 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262de1c-3913-43f5-a3ea-7bcb03536afa admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.614200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.614891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.615189 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.615619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.616725 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.619457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.638171 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1435/2850] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:15.643977 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.658512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e70a3-3337-451f-8001-59dd2b9fa9fa admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.665001 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.665971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.666366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.666749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.667714 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.670458 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5925286-8237-44ae-a4bf-8e1a79adb007 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.690345 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1416/2851] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:15.695154 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.706264 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9496-5fad-42d7-a630-75da9f044ceb admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.711075 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.711803 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.712083 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.712351 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] PATH_INFO: `/v3/projects/47196dc5cc824ce49dd5294fc59d50c4/users/1b9f88ab5ea341b8995aae4262ff041a/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.733800 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47196dc5cc824ce49dd5294fc59d50c4, user_id=1b9f88ab5ea341b8995aae4262ff041a, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.738305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.759543 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1436/2852] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:15 2022] PUT /identity/v3/projects/47196dc5cc824ce49dd5294fc59d50c4/users/1b9f88ab5ea341b8995aae4262ff041a/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:19:15.767941 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f939-1d9b-4f8b-958b-6e05effb8a0b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.768412 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f939-1d9b-4f8b-958b-6e05effb8a0b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.768907 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f939-1d9b-4f8b-958b-6e05effb8a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.796996 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-bf75f939-1d9b-4f8b-958b-6e05effb8a0b None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:15.798997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-bf75f939-1d9b-4f8b-958b-6e05effb8a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:15.875651 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1417/2853] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:15.888438 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:15.890782 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:15.956718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd7dc0-b9db-4dd2-b81b-055e2b48faf4 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:15.960697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:15.961316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:15.961667 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:15.961960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:15.972509 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:15.974034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:15.986152 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1437/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:21.109307 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.110195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.110195 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.158909 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] MFA Rules not processed for user `1b9f88ab5ea341b8995aae4262ff041a`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:21.161250 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:21.170034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:21.265615 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1418/2855] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:21.271624 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.286176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27386c56-6844-464a-9c76-89d907883b9f service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.291263 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.291891 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.292147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.292416 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.293059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.295062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.301430 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:21.345921 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1438/2856] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:19:21 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:21.350209 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.371862 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99dd20-0a5b-4a91-8bf5-84c302cbe74b None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.467674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.468204 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.468500 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.469382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.469382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.472114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.508309 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1419/2857] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:19:21 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:19:21.513406 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.542997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003f365-04e2-4bcd-8ed2-64791f582cdf admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.554502 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.555632 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.556042 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.556516 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.557825 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.560593 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.589117 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1439/2858] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:21.595362 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.610090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692abba-d362-4727-afc3-b02496b6ffb7 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.618686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.619717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.620091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.620487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] PATH_INFO: `/v3/projects/23402f167ff3456faa86417d13206c5d/users/e0fdda6f409247c3ad4b9f8087bdbfb1/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.648043 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23402f167ff3456faa86417d13206c5d, user_id=e0fdda6f409247c3ad4b9f8087bdbfb1, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.652037 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.678791 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1420/2859] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:21 2022] PUT /identity/v3/projects/23402f167ff3456faa86417d13206c5d/users/e0fdda6f409247c3ad4b9f8087bdbfb1/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:19:21.684392 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.699862 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29955789-2ccd-4a04-85db-f95249c1125e admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.782124 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.782772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.783058 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.783291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.783939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.785627 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897d442-6571-4f17-8897-30e9b526fe68 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.797487 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1440/2860] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:19:21.802035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:21.819016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bd1a4-bcd1-4fee-9c44-e49389b055d9 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:21.826090 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:21.827100 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:21.827557 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:21.828018 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] PATH_INFO: `/v3/projects/23402f167ff3456faa86417d13206c5d/users/e0fdda6f409247c3ad4b9f8087bdbfb1/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:21.830809 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23402f167ff3456faa86417d13206c5d, user_id=e0fdda6f409247c3ad4b9f8087bdbfb1, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:21.835481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:21.851049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1421/2861] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:19:21 2022] PUT /identity/v3/projects/23402f167ff3456faa86417d13206c5d/users/e0fdda6f409247c3ad4b9f8087bdbfb1/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:19:26.622050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894f92-9607-421a-990a-e01047ca4f1a None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:26.622583 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894f92-9607-421a-990a-e01047ca4f1a None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:26.622882 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f894f92-9607-421a-990a-e01047ca4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:26.656691 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1f894f92-9607-421a-990a-e01047ca4f1a None None] MFA Rules not processed for user `e0fdda6f409247c3ad4b9f8087bdbfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:26.657943 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1f894f92-9607-421a-990a-e01047ca4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:26.730590 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1441/2862] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:26.746319 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:26.749626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:26.811200 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767915-9990-43c7-b928-519c8a984978 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:26.815282 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f03a7d5287c48e4821d308767f4c141', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:26.815905 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:26.816176 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:26.816439 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:26.829434 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:26.880476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:26.881910 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:26.891316 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1422/2863] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:19:27.024815 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c6e0-7c76-44f1-9042-bb9835794719 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:27.025100 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c6e0-7c76-44f1-9042-bb9835794719 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:27.025368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c6e0-7c76-44f1-9042-bb9835794719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:27.045251 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-5069c6e0-7c76-44f1-9042-bb9835794719 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:27.046547 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5069c6e0-7c76-44f1-9042-bb9835794719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:27.111482 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1442/2864] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:19:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:27.162956 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:27.164392 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:27.238294 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c6400-4c1c-4260-8e26-adcbdb9493dd service glance] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:27.243639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:27.244290 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:27.244558 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:27.244823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:27.253836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:27.255406 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80829e9b-07ed-45a6-90e8-29d4b99aeb3f service swift] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:27.268248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1423/2865] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:19:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:19:35.690280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:35.690844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:35.690844 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:35.698924 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:35.712367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] MFA Rules not processed for user `1b9f88ab5ea341b8995aae4262ff041a`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:35.713564 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:35.720726 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1443/2866] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:19:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:37.262227 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=53 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:37.714331 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:37.714736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:37.715072 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:37.735633 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:19:37.736954 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:37.744848 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1424/2867] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 17 20:19:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:19:37.758724 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:37.771785 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59a95-5f99-4197-8be6-b2b860cb0122 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:37.777649 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:37.778284 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:37.778556 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:37.778823 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:37.788156 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:37.789799 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:37.799079 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1444/2868] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:19:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:19:47.729582 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:47.795236 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=8 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:19:57.938198 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:19:57.939762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:58.066947 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b361442-5199-4c3c-877c-bc48c14805c1 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:19:58.074277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:19:58.075626 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:19:58.076102 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:19:58.076446 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:19:58.078781 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:19:58.094773 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:19:58.096380 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:19:58.109149 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1425/2869] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:19:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:20:01.895860 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:01.956563 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ba0e4-95ba-4e9c-9fe9-6848c0eaa5bb service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:01.960147 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:01.960718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:01.960994 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:01.961256 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:01.963054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:02.029721 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:02.031211 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:02.041092 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1445/2870] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:20:07.623521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:07.625162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:07.637671 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad37b9-6a49-4ef2-8b8c-cae5f5600ca6 service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:07.641827 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:07.642648 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:07.642969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:07.643330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:07.654771 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:07.656162 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:07.668983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1426/2871] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:20:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:20:09.236669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:09.239026 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:09.321563 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:09.324190 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:09.349526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ccf8-ef1f-4b5d-b83b-f6d56531829e service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:09.350731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c75446-d89d-4de7-b9ac-ffd43554cfbc service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:09.354661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:09.355460 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:09.355956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:09.356270 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:09.358337 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:09.361943 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:09.363716 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:09.364496 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:09.365453 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] PATH_INFO: `/v3/users/112f1de9bcdc4b8fb3d402190d4f0751` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:09.367841 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=112f1de9bcdc4b8fb3d402190d4f0751)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:09.371595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:09.372814 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:09.373274 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:09.394772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1446/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:20:09.444150 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112f1de9bcdc4b8fb3d402190d4f0751'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.444834 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112f1de9bcdc4b8fb3d402190d4f0751'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.445849 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112f1de9bcdc4b8fb3d402190d4f0751'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.467170 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112f1de9bcdc4b8fb3d402190d4f0751'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.489746 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112f1de9bcdc4b8fb3d402190d4f0751'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.509076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1427/2873] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:20:09 2022] DELETE /identity/v3/users/112f1de9bcdc4b8fb3d402190d4f0751 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:20:09.713887 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:09.728869 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336ca2-ff10-4eaa-a759-d5bbd8c194b9 service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:09.782280 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:20:09.809388 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:09.810244 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:09.810797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:09.811219 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] PATH_INFO: `/v3/projects/458eee750aa741c493481ee785761172` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:09.812883 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] RBAC: Authorizing `identity:delete_project(project_id=458eee750aa741c493481ee785761172)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:09.816050 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:09.879932 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458eee750aa741c493481ee785761172', 'request_id': 'req-af3df8ec-eac0-4565-b0af-98f02597746f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.887186 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458eee750aa741c493481ee785761172', 'request_id': 'req-af3df8ec-eac0-4565-b0af-98f02597746f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.894660 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458eee750aa741c493481ee785761172', 'request_id': 'req-af3df8ec-eac0-4565-b0af-98f02597746f'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.914740 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] The token cache is being invalidate because project 458eee750aa741c493481ee785761172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:20:09.915303 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.916123 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:09.917983 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1447/2874] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:20:09 2022] DELETE /identity/v3/projects/458eee750aa741c493481ee785761172 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:20:09.922147 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:09.923731 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:09.938002 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438d467-ebf4-4cd3-93a0-b9f0f60a5f3f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:10.006604 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:20:10.027400 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:10.028067 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:10.028373 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:10.028680 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] PATH_INFO: `/v3/users/66e6af15e4144be4ad1560e7216b1aa3` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:10.029898 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=66e6af15e4144be4ad1560e7216b1aa3)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:10.031845 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:10.112843 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66e6af15e4144be4ad1560e7216b1aa3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.114044 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66e6af15e4144be4ad1560e7216b1aa3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.115121 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66e6af15e4144be4ad1560e7216b1aa3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.143155 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66e6af15e4144be4ad1560e7216b1aa3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.152681 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66e6af15e4144be4ad1560e7216b1aa3'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.181574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1428/2875] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:20:09 2022] DELETE /identity/v3/users/66e6af15e4144be4ad1560e7216b1aa3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:20:10.592193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:20:10.596071 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:20:10.619368 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3df8ec-eac0-4565-b0af-98f02597746f admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:20:10.715806 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:20:10.716596 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:20:10.716909 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:20:10.717281 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] PATH_INFO: `/v3/projects/9d1f911a05ac422487d9b8fb15732f25` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:20:10.719055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d1f911a05ac422487d9b8fb15732f25)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:20:10.721234 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:20:10.794614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d1f911a05ac422487d9b8fb15732f25', 'request_id': 'req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.801200 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d1f911a05ac422487d9b8fb15732f25', 'request_id': 'req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.808604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d1f911a05ac422487d9b8fb15732f25', 'request_id': 'req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.822980 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:20:10.833694 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] The token cache is being invalidate because project 9d1f911a05ac422487d9b8fb15732f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:20:10.834210 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.836424 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:20:10.837610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1448/2876] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:20:10 2022] DELETE /identity/v3/projects/9d1f911a05ac422487d9b8fb15732f25 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:20:20.168574 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:10.961130 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:10.962789 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:11.115284 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7992a6-ed5b-4633-8d7a-444e1ed2ee5e admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:11.184163 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:11.185159 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:11.185673 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:11.186161 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:11.189201 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:11.209472 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:11.212142 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676b50d-a9c1-46b1-b6fb-808a28182836 service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:11.228096 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1429/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:22:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:22:21.222071 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=89 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:34.926377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:34.928291 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:35.065840 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e16b74-7d0c-4b33-8f74-edd2cf1d82a5 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:35.074112 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:35.075029 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:35.075614 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:35.076054 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:35.079772 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:35.098363 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:35.102441 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:35.123097 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1449/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:22:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:22:42.479218 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:42.479944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:42.479944 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:42.519034 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] MFA Rules not processed for user `585bda89775640bfb817ffc41525181d`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:22:42.520277 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:42.591326 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1430/2879] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 17 20:22:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:22:42.637830 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:42.639359 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:42.724230 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaa01e-21df-48fc-b593-9c3f6b8da67e service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:42.730507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '103c6c2fb3104914b8b4ec7c196f4996', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:42.731121 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:42.731394 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:42.731661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:42.741093 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:42.742730 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:42.751481 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1450/2880] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 17 20:22:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 17 20:22:50.451538 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:50.454107 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:50.574431 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706557f-57f2-4c7e-921f-6081791003d0 None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:50.579669 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:50.580325 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:50.580600 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:50.580873 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] PATH_INFO: `/v3/users/1b9f88ab5ea341b8995aae4262ff041a` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:50.582187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9f88ab5ea341b8995aae4262ff041a)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:50.584154 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:50.655761 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9f88ab5ea341b8995aae4262ff041a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:50.656454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9f88ab5ea341b8995aae4262ff041a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:50.657181 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9f88ab5ea341b8995aae4262ff041a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:50.670992 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9f88ab5ea341b8995aae4262ff041a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:50.676476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:50.682976 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9f88ab5ea341b8995aae4262ff041a'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:50.700118 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1431/2881] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:22:50 2022] DELETE /identity/v3/users/1b9f88ab5ea341b8995aae4262ff041a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:50.903905 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:50.916013 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94472-5ace-4f7c-acaf-dfda69ba9c71 service swift] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:50.971584 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=100 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:50.991680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:50.992663 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:50.992939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:50.993290 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] PATH_INFO: `/v3/projects/47196dc5cc824ce49dd5294fc59d50c4` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:50.995090 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47196dc5cc824ce49dd5294fc59d50c4)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:50.997049 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:51.051312 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47196dc5cc824ce49dd5294fc59d50c4', 'request_id': 'req-b9f27700-785d-4bff-b69c-9ce92f98db21'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.058127 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47196dc5cc824ce49dd5294fc59d50c4', 'request_id': 'req-b9f27700-785d-4bff-b69c-9ce92f98db21'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.065332 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47196dc5cc824ce49dd5294fc59d50c4', 'request_id': 'req-b9f27700-785d-4bff-b69c-9ce92f98db21'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.082927 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] The token cache is being invalidate because project 47196dc5cc824ce49dd5294fc59d50c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:22:51.083223 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.084305 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.086948 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1451/2882] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:22:50 2022] DELETE /identity/v3/projects/47196dc5cc824ce49dd5294fc59d50c4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:51.091415 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:51.093266 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:51.107372 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80eec6-0e56-4cd2-b52a-60349599354f admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:51.194499 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:51.195061 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.195312 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.195585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] PATH_INFO: `/v3/users/e0fdda6f409247c3ad4b9f8087bdbfb1` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.196790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fdda6f409247c3ad4b9f8087bdbfb1)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:51.198736 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:51.269426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fdda6f409247c3ad4b9f8087bdbfb1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.270122 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fdda6f409247c3ad4b9f8087bdbfb1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.270796 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fdda6f409247c3ad4b9f8087bdbfb1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.284517 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fdda6f409247c3ad4b9f8087bdbfb1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.291865 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-f1048068-1821-4fc0-b05f-8cd6aa7a11e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fdda6f409247c3ad4b9f8087bdbfb1'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.308352 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1432/2883] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:22:51 2022] DELETE /identity/v3/users/e0fdda6f409247c3ad4b9f8087bdbfb1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:51.535842 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:51.537953 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:51.564523 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27700-785d-4bff-b69c-9ce92f98db21 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:51.647604 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:51.648246 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.648521 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.648790 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] PATH_INFO: `/v3/projects/23402f167ff3456faa86417d13206c5d` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.649991 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=23402f167ff3456faa86417d13206c5d)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:51.651911 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:51.702832 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23402f167ff3456faa86417d13206c5d', 'request_id': 'req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.709580 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23402f167ff3456faa86417d13206c5d', 'request_id': 'req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.716469 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23402f167ff3456faa86417d13206c5d', 'request_id': 'req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.729589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] The token cache is being invalidate because project 23402f167ff3456faa86417d13206c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81533) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:22:51.729904 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.730591 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-f5d37b49-b1b4-4cfb-b116-3620c8e074ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:22:51.732191 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1452/2884] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:22:51 2022] DELETE /identity/v3/projects/23402f167ff3456faa86417d13206c5d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:51.738287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.739152 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.739340 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.818247 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:22:51.819790 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:51.855258 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:51.895476 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1433/2885] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:22:51.903603 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.903847 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.904131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.913618 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:51.926589 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:22:51.928411 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:51.937115 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1453/2886] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:22:51.943543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:51.954686 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993b54e-86e3-4671-abb3-f27c2effd48a None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:51.960177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:51.960825 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.961093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.961355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.962049 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81532) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 17 20:22:51.962321 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:51.963942 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:51.976639 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1434/2887] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 17 20:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 17 20:22:51.981420 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:51.991277 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c316963-e84a-44db-955e-1e330bae3312 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:51.996272 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:51.996898 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:51.997170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:51.997426 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:51.998181 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.000088 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.040826 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1454/2888] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:22:51 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:22:52.045521 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:52.059430 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2dcb-639b-4ac1-b69e-f63a2cfd5b45 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:52.133778 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:52.134701 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.135134 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.135561 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.136527 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.139410 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.165348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1435/2889] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:22:52 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 17 20:22:52.170791 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:52.191708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f13213-f2dd-42e7-adc6-7186bb56c000 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:52.203698 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:52.204595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.204595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.205447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.205447 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.207484 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.222243 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1455/2890] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 17 20:22:52.226982 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:52.241129 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a140254-1837-4942-83a1-a80084529e3b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:52.247298 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:52.248330 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.248697 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.248997 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.249823 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.252003 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1644f1-e95b-410f-8f04-299d34435e40 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.267353 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1436/2891] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:22:52.271804 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:52.283577 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b508-e6e0-479d-8fe4-c1e90b4e25a7 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:52.289335 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:52.290032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.290306 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.290659 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] PATH_INFO: `/v3/projects/e7ecb49247654d5f92cf32c4fbb01845/users/e0240700f55f4376a51483fd0f15dd48/roles/8f4a64d3f31545619fe7a9e422d55398` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.307855 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ecb49247654d5f92cf32c4fbb01845, user_id=e0240700f55f4376a51483fd0f15dd48, role_id=8f4a64d3f31545619fe7a9e422d55398)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.310731 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.326231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1456/2892] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:22:52 2022] PUT /identity/v3/projects/e7ecb49247654d5f92cf32c4fbb01845/users/e0240700f55f4376a51483fd0f15dd48/roles/8f4a64d3f31545619fe7a9e422d55398 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:52.332007 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c301014-0ccd-4f8d-b4f3-a7ca3226cbbc None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.332490 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c301014-0ccd-4f8d-b4f3-a7ca3226cbbc None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.332875 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c301014-0ccd-4f8d-b4f3-a7ca3226cbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.353483 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-3c301014-0ccd-4f8d-b4f3-a7ca3226cbbc None None] MFA Rules not processed for user `3a9bd68510704bb0a6eea1e00548849b`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:22:52.355208 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-3c301014-0ccd-4f8d-b4f3-a7ca3226cbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:52.419927 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1437/2893] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:22:52.431586 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:52.434228 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:52.532727 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16d9d9-38b7-4b85-9e9e-4fd4610c4c6c admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:52.537138 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:52.537769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:52.538033 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:52.538295 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:52.549319 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:52.551166 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:52.562971 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1457/2894] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:22:57.438727 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:57.439457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:57.439457 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:57.474819 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] MFA Rules not processed for user `e0240700f55f4376a51483fd0f15dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:22:57.477236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:22:57.531614 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:57.595327 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1438/2895] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:22:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:22:57.599708 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:57.613308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad54a82-00e6-434d-a6de-c0630fac91dc service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:57.621768 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:57.622397 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:57.622672 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:57.623011 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:57.623651 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:57.625874 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:57.669956 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1458/2896] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 17 20:22:57 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:22:57.671585 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:57.683961 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90212fd0-7355-4b95-89f9-8baf3337514f None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:57.767375 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:57.768089 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:57.768366 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:57.768638 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] PATH_INFO: `/v3/users` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:57.769426 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:57.772428 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:57.803124 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1439/2897] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 17 20:22:57 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 17 20:22:57.808620 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:57.825373 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f514c19-9956-4006-9b5c-1d63fed7de69 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:57.832268 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:57.833030 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:57.833314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:57.833805 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:57.834498 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:57.836759 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:57.849077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1459/2898] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:22:57.854182 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:57.872504 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c904c-831a-4132-b626-d0549c3d148b admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:57.877830 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:57.878753 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:57.879114 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:57.879747 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] PATH_INFO: `/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa/users/276fc50884e04f1c877418efd53933ea/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:57.899738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=757bb1f073544ea6b7deefe8b7ad4baa, user_id=276fc50884e04f1c877418efd53933ea, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:57.902737 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:57.919348 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1440/2899] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:22:57 2022] PUT /identity/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa/users/276fc50884e04f1c877418efd53933ea/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:22:57.924581 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:57.938308 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ec212-db2d-4840-b3a5-44d0cee8e861 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:57.990889 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:22:58.008756 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:58.009367 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:58.009669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:58.009936 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:58.010585 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:58.012194 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf9326-0ced-4cb9-820f-50d6fa411b38 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:58.023454 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1460/2900] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 17 20:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 17 20:22:58.027836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:22:58.039928 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de79ae-300b-48ec-8b46-bab408e329b6 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:22:58.044674 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340862-282a-491b-b969-1d792441381a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:22:58.045456 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340862-282a-491b-b969-1d792441381a admin admin] REQUEST_METHOD: `PUT` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:22:58.045939 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340862-282a-491b-b969-1d792441381a admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:22:58.046212 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7340862-282a-491b-b969-1d792441381a admin admin] PATH_INFO: `/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa/users/276fc50884e04f1c877418efd53933ea/roles/c974a286ee1942c8bab82b232c6af142` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:22:58.048225 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7340862-282a-491b-b969-1d792441381a admin admin] RBAC: Authorizing `identity:create_grant(project_id=757bb1f073544ea6b7deefe8b7ad4baa, user_id=276fc50884e04f1c877418efd53933ea, role_id=c974a286ee1942c8bab82b232c6af142)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:22:58.050983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7340862-282a-491b-b969-1d792441381a admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:22:58.063048 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1441/2901] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 17 20:22:58 2022] PUT /identity/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa/users/276fc50884e04f1c877418efd53933ea/roles/c974a286ee1942c8bab82b232c6af142 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 17 20:23:02.971595 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5bbef-6a7e-44b9-824c-8fd268aa0650 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:02.972193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5bbef-6a7e-44b9-824c-8fd268aa0650 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:02.972193 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5bbef-6a7e-44b9-824c-8fd268aa0650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:03.018267 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-4ae5bbef-6a7e-44b9-824c-8fd268aa0650 None None] MFA Rules not processed for user `276fc50884e04f1c877418efd53933ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:23:03.019613 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4ae5bbef-6a7e-44b9-824c-8fd268aa0650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:03.124094 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1461/2902] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:23:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:23:03.140590 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340862-282a-491b-b969-1d792441381a admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:03.143291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a7340862-282a-491b-b969-1d792441381a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:03.290529 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:03.299762 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7340862-282a-491b-b969-1d792441381a admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:03.309394 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:03.310097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:03.310418 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:03.310685 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:03.388047 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:03.389660 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:03.407833 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1442/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:23:13.120707 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=44 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:13.395822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=27 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:25.839475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:25.840103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:25.840103 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:25.861282 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] MFA Rules not processed for user `e0240700f55f4376a51483fd0f15dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:23:25.862566 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:25.870669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1462/2904] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:23:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:23:25.913462 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:25.915023 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:25.996512 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771174e-6c97-4782-bd42-47c68cd99a1a service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:26.000497 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:26.001117 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:26.001389 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:26.001704 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:26.003552 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:26.017782 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:26.019199 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:26.030477 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1443/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:26.326872 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:26.328437 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:26.384176 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcd4ed-891a-405d-9784-3b553d89f508 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:26.390839 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:26.391497 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:26.391769 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:26.392034 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:26.394356 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:26.405572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:26.407101 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:26.416635 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1463/2906] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:26.895677 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:26.898093 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:26.920257 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:26.929687 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158f168-c8b7-4835-b796-2a763e3a8734 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:26.935032 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:26.935630 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:26.935915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:26.936155 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:26.938037 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:26.952259 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:26.953987 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:26.965572 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1464/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:26.979273 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d072fa-d74d-4d1d-a54b-afdd2bea3b33 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:26.984631 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:26.985355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:26.985719 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:26.985895 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:26.987879 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:27.000650 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:27.002279 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:27.014230 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1444/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:36.960222 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=35 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:37.009591 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=67 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:40.572221 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:40.575314 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:40.591353 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054178b1-d26b-4967-93e4-4dd723d23874 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:40.597008 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:40.597666 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:40.597923 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:40.598184 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:40.599945 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:23:40.613399 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:40.614963 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:40.626550 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1465/2909] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:44.051523 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:23:44.062566 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e393570-bc22-4874-b034-189544f1f7fb service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:23:44.067969 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:23:44.068592 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:23:44.068867 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:23:44.069231 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:23:44.081580 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:23:44.083035 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581add45-f548-413e-9deb-abbcc2a3e414 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:23:44.093859 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1445/2910] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:23:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:23:50.621434 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:23:54.088738 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:24:11.890102 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:24:11.902514 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff20a5-8457-4716-869e-f1f8a67de703 service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:24:11.908151 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:24:11.909110 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:24:11.909387 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:24:11.909703 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:24:11.912377 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:24:11.929877 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:24:11.932797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:24:11.946449 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1466/2911] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:24:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:24:21.939866 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:26:04.999215 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] REQUEST_METHOD: `POST` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:26:04.999831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:26:04.999831 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:26:05.019248 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.auth.core [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] MFA Rules not processed for user `e0240700f55f4376a51483fd0f15dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=81532) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:26:05.020465 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:26:05.027603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1446/2912] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:26:05.613080 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:26:05.622619 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fa1d-3948-43dd-a93a-8dfe46c938cb service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:26:05.626640 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:26:05.627249 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:26:05.627526 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:26:05.627797 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:26:05.629763 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:26:05.640051 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:26:05.641519 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:26:05.650260 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1467/2913] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:26:15.013688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=3 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:26:15.646235 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:26:24.980038 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:26:24.996432 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efc2df-ddd0-4d4b-bcc2-8979e582d14c None None] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:26:25.003354 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:26:25.004084 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:26:25.004420 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:26:25.004930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:26:25.006919 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:26:25.020822 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:26:25.023971 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:26:25.036930 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1447/2914] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:26:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:26:35.031545 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:27:18.188248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:27:18.202443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e15eb-7b91-4e97-b90c-17518ef0c13b service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:27:18.208047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:27:18.208669 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:27:18.208942 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:27:18.209207 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:27:18.211146 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:27:18.224153 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:27:18.225885 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04943ee6-bf43-44e2-8462-02856d078eb7 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:27:18.236939 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1468/2915] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:27:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:27:28.231960 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:28:11.981503 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:28:11.983543 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:28:11.996460 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d1b30-d62e-4478-90df-4553af0489aa service nova] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:28:12.001387 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97be7f3685b44cfa26aeae0f5b5d931', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:28:12.002059 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:28:12.002291 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:28:12.002581 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:28:12.012419 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:28:12.013883 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:28:12.024752 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1448/2916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:28:22.019027 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=6 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:28:23.173089 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] REQUEST_METHOD: `POST` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:28:23.174025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:28:23.174025 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:28:23.195157 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.auth.core [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] MFA Rules not processed for user `e0240700f55f4376a51483fd0f15dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=81533) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 17 20:28:23.196796 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:28:23.205969 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1469/2917] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 17 20:28:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 17 20:28:26.285824 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:28:26.296153 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c447b5e-40ac-444b-a49b-c71a15f70490 service cinder] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:28:26.300236 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:28:26.300950 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:28:26.301484 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:28:26.301882 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:28:26.314355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:28:26.315978 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:28:26.328292 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1449/2918] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:28:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:28:26.665495 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:28:26.681170 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a0c61-047f-44c7-a10d-9fde8ba13a57 None None] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:28:26.687835 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1244993f093f4ad0920e3a42725a2308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:28:26.688455 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:28:26.688728 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:28:26.688995 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:28:26.691055 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:28:26.706062 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:28:26.707020 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:28:26.720383 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1470/2919] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:28:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:28:28.711872 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:28:28.721223 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1b17b-0079-4ef0-ac87-5f7be4054218 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:28:28.724970 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:28:28.725603 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:28:28.725881 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:28:28.726151 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:28:28.737684 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:28:28.739123 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:28:28.750749 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1450/2920] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:28:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:28:36.713718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=9 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:28:38.745646 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=12 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:29:13.865198 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:29:13.876178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9ff41-38f3-4df7-8315-5fa5c90d158a service nova] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:29:13.880764 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:29:13.881393 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:29:13.881718 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:29:13.881986 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:29:13.891729 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:29:13.893196 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:29:13.902575 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1471/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:29:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:29:23.898396 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=6 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:30:59.561424 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:30:59.578653 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7effd94-4e55-49a1-9389-947d8a301c77 service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:30:59.583445 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4eae163ceee54be9a10f7475ab87392e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:30:59.584168 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:30:59.584562 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:30:59.585016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:30:59.661869 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:30:59.663287 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:30:59.674016 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1451/2922] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:31:02.965610 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:02.986972 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fe4f-1065-4148-873c-790161a22020 service neutron] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:02.993457 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:02.994131 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:02.994410 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:02.994680 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:02.996813 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:31:03.010917 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:03.012778 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:03.025077 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1472/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 17 20:31:06.492202 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:06.509949 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41e64e-e136-4676-ba8d-bc9956da666a service neutron] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:06.516675 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:06.517467 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] REQUEST_METHOD: `GET` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:06.517870 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:06.518222 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:06.520441 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:31:06.534265 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:06.536877 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:06.554507 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:06.568415 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24ea23-4fdc-4ef6-9280-9b37d1bd6b1b service placement] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:06.574952 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1452/2924] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:31:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:31:06.578637 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:06.579757 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:06.580237 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:06.580695 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] PATH_INFO: `/v3/users/e0240700f55f4376a51483fd0f15dd48` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:06.582809 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0240700f55f4376a51483fd0f15dd48)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:06.587992 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:06.667886 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0240700f55f4376a51483fd0f15dd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:06.680333 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0240700f55f4376a51483fd0f15dd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:06.701712 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0240700f55f4376a51483fd0f15dd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:06.711587 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0240700f55f4376a51483fd0f15dd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:06.712231 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0240700f55f4376a51483fd0f15dd48'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:06.723297 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1473/2925] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:31:06 2022] DELETE /identity/v3/users/e0240700f55f4376a51483fd0f15dd48 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:31:06.943811 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:06.957605 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73313dc-e9d5-42b5-bf21-35214be42bef service placement] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:07.039679 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:07.040507 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:07.040836 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:07.041146 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] PATH_INFO: `/v3/projects/e7ecb49247654d5f92cf32c4fbb01845` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:07.042501 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ecb49247654d5f92cf32c4fbb01845)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:07.044540 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:07.111449 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ecb49247654d5f92cf32c4fbb01845', 'request_id': 'req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.117454 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ecb49247654d5f92cf32c4fbb01845', 'request_id': 'req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.127271 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ecb49247654d5f92cf32c4fbb01845', 'request_id': 'req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.140097 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] The token cache is being invalidate because project e7ecb49247654d5f92cf32c4fbb01845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:31:07.140390 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.141076 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.142688 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1453/2926] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:31:06 2022] DELETE /identity/v3/projects/e7ecb49247654d5f92cf32c4fbb01845 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:31:07.147475 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:07.149047 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:31:07.164287 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2483f-d487-4305-b692-f715cd16dcb1 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:07.236966 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:07.237645 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:07.237915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:07.238178 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] PATH_INFO: `/v3/users/276fc50884e04f1c877418efd53933ea` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:07.239347 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] RBAC: Authorizing `identity:delete_user(user_id=276fc50884e04f1c877418efd53933ea)` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:07.241293 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:07.299848 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276fc50884e04f1c877418efd53933ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.315533 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276fc50884e04f1c877418efd53933ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.320513 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:31:07.330959 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276fc50884e04f1c877418efd53933ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.340386 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276fc50884e04f1c877418efd53933ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.341059 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.notifications [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276fc50884e04f1c877418efd53933ea'} {{(pid=81533) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.347486 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1474/2927] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 17 20:31:07 2022] DELETE /identity/v3/users/276fc50884e04f1c877418efd53933ea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:31:07.561728 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Authenticating user token {{(pid=81532) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:31:07.563698 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.fernet_utils [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81532) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:31:07.577382 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad02-6c1b-4bc2-8fbc-81ec83177d63 admin admin] Validating token access rules against request {{(pid=81532) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:31:07.587837 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:31:07.655187 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a9bd68510704bb0a6eea1e00548849b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '79e7870848b34ea38cb8567a241b0ce4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81532) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:31:07.655717 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:31:07.655985 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:31:07.656253 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] PATH_INFO: `/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa` {{(pid=81532) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:31:07.657413 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757bb1f073544ea6b7deefe8b7ad4baa)` {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:31:07.659355 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] RBAC: Authorization granted {{(pid=81532) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:31:07.725177 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757bb1f073544ea6b7deefe8b7ad4baa', 'request_id': 'req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.732983 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757bb1f073544ea6b7deefe8b7ad4baa', 'request_id': 'req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.745481 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757bb1f073544ea6b7deefe8b7ad4baa', 'request_id': 'req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6'} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.760091 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] The token cache is being invalidate because project 757bb1f073544ea6b7deefe8b7ad4baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81532) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 17 20:31:07.760487 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.761341 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG keystone.notifications [None req-49b9f7e3-210d-40c0-a7b6-fd8108b3edf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81532) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 17 20:31:07.763741 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: [pid: 81532|app: 0|req: 1454/2928] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 17 20:31:07 2022] DELETE /identity/v3/projects/757bb1f073544ea6b7deefe8b7ad4baa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 17 20:31:17.343997 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:31:17.756232 dv-cinder-820027-19-iscsi devstack@keystone.service[81532]: DEBUG dbcounter [-] [81532] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81532) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 17 20:32:22.146661 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Authenticating user token {{(pid=81533) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 17 20:32:22.148316 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:32:22.275952 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16116a9e-6aff-44b2-867b-8c1094774815 admin admin] Validating token access rules against request {{(pid=81533) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 17 20:32:22.348076 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7abc4e612c0241ac8efb2789a5741250', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '327ecf4949a040998a1557337864548d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81533) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 17 20:32:22.348697 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] REQUEST_METHOD: `GET` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 17 20:32:22.348973 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] SCRIPT_NAME: `/identity` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 17 20:32:22.349248 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81533) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 17 20:32:22.351169 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.fernet_utils [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81533) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 17 20:32:22.363820 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 17 20:32:22.365915 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0130c8-0f5a-4efc-9f5f-baa23c0c4665 service placement] RBAC: Authorization granted {{(pid=81533) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 17 20:32:22.377443 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: [pid: 81533|app: 0|req: 1475/2929] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 17 20:32:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 17 20:32:32.372568 dv-cinder-820027-19-iscsi devstack@keystone.service[81533]: DEBUG dbcounter [-] [81533] Writing DB stats keystone:SELECT=78 {{(pid=81533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}